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Preface  and  Acknowledgements 

One  hundred  years  ago,  William  H.  Allen — of  the  then-newly  founded  Bureau  of 
Municipal  Research — issued  a  call  in  the  Journal  of  Accountancy  for  “1 ,000  accountants  for 
municipal  research.”  Reflecting  the  Progressive  Era’s  focus  on  domestic  reform  after  the 
closing  of  the  American  frontier,  Allen  wrote: 

Only  a  pessimist  will  believe  that  the  day  is  past  for  the  pioneer.  It  is 
true  that  America  has  been  discovered  and  that  the  law  of  diminishing  returns 
long  since  began  to  operate  in  the  gold  fields  of  California,  the  wheat  fields  of 
the  Northwest  and  the  oil  wells  of  Pennsylvania.  It  is  also  true  that  there  is 
less  opportunity  today  than  ever  before  for  adventure  of  the  story  book  type. 

But  to  young  men  [sic]  capable  of  thrilling  with  excitement  when  confronted 
with  new  problems  to  solve  and  new  ideas  to  work  out,  I  wish  seriously  to 
recommend  a  substitute  for  the  North  Pole — the  unexploited  field  of  municipal 
accounting  and  municipal  business.1 

Allen’s  call  followed  the  Bureau’s  early  and  remarkable  successes,  both  in  exposing 
waste  and  corruption  in  New  York  City’s  government  and  in  devising  and  installing 
managerial  systems  for  increased  efficiency  and  transparency: 

[T]he  mayor,  comptroller,  commissioner  of  street  cleaning,  president 
of  Bellevue  and  allied  hospitals,  commissioner  of  parks  and  the 
commissioners  of  accounts  have  requested  cooperation,  and  used 
departmental  facilities  and  men  [sic]  for  research  and  reorganization.  We 
believe  that  similar  cooperation  will  be  obtained  wherever  private  bodies  or 
especially  trained  accountants  approach  the  problem  of  municipal  business 
with  the  sole  motive  of  advancing  the  interests  of  the  general  public,  and  not 
with  a  desire  to  do  sharpshooting,  to  turn  up  a  scandal  or  to  turn  out  the 
rascals.2 

History  has  not  recorded  the  extent  to  which  Allen’s  call  for  1 ,000  accountants  was 
answered.  History  has  recorded,  however,  the  very  clear  and  significant  contributions  of  his 
Bureau’s  work — particularly  its  research  agenda — to  the  formation  of  the  field  of  American 
Public  Administration.3  Thus,  while  Allen  certainly  perceived  the  potential  contributions  of 
administrative  research,  it’s  highly  doubtful  he  could  have  imagined  the  development  and 
maturation  over  the  next  century  of  this  entirely  new  field  of  study  in  the  US.  Public 
Administration  today  includes  hundreds  of  graduate  degree  programs,  dozens  of  academic 
journals  and  conferences,  and  thousands  of  scholars.  The  objects  of  its  study  have 
expanded  from  municipal  administration  to  include  federal,  state,  international,  and,  more 
recently,  not-for-profit  administration. 


1  Allen,  W.H.  (1908,  July).  Wanted — One  thousand  efficient  accountants  for  municipal  research. 
Journal  of  Accountancy,  6, 187. 

2  Allen,  1908,  pp.  192-193. 

3  Waldo,  D.  (1984).  The  administrative  state  (2nd  ed.).  New  York:  Holmes  &  Meier,  p.  31 . 
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Today,  we  issue  a  call  for  “1 ,000  scholars  for  defense  acquisition  research.”  As 
Allen  believed  in  the  possibilities  for  municipal  research,  we  also  believe  in  the  possibilities 
for  melioration  of  acquisition’s  seemingly  intractable  problems  through  systematic  study. 
While  Allen  saw  the  skills  of  accountants  as  sufficient  for  the  tasks  he  had  in  mind,  we 
instead  call  for  a  truly  interdisciplinary  mix  of  scholars  suitable  for  engaging  the  diverse 
facets  of  acquisition’s  many  technical,  managerial,  and  political  issues. 

Obviously,  such  an  ambitious  call  cannot  be  answered  by  a  single  or  even  a  dozen 
institutions.  Accordingly,  the  NPS  Acquisition  Research  Program  has  among  its  principal 
objectives  the  cultivation  of  an  interdisciplinary  community  of  acquisition  scholars  from  many 
institutions  around  the  world.  This  Symposium  is  merely  a  single  step  toward  achieving  that 
objective.  Other  recent  steps  include  new  research  partnerships  that  NPS  has  forged  with 
several  other  universities  and  the  new  International  Journal  of  Defense  Acquisition 
Management  (http://www.acquisitionjournal.org),  a  scholarly  journal  jointly  published  and 
supported  by  the  Acquisition  Research  Program  and  Cranfield  University  at  the  Defence 
College  of  Management  of  Technology. 

From  our  limited  perspective,  such  steps  may  seem  woefully  inadequate  for  the  task 
of  achieving  meaningful  and  lasting  acquisition  reform.  If  so,  we  may  do  well  to  look  forward 
into  the  next  century  and  imagine  our  intellectual  descendents  who  will  study  in  a  fully 
mature  field  of  defense  acquisition  management  and  who  will  commend  us  for  our  efforts. 

We  gratefully  acknowledge  the  ongoing  support  and  leadership  of  our  sponsors, 
whose  foresight  and  vision  have  assured  the  continuing  success  of  the  Acquisition 
Research  Program: 

•  Under  Secretary  of  Defense  (Acquisition,  Technology  and  Logistics) 

•  Program  Executive  Officer  (Ships) 

•  Program  Executive  Officer  (Integrated  Warfare  Systems) 

•  Program  Executive  Officer  (Littoral  and  Mine  Warfare) 

•  Commander,  Naval  Sea  Systems  Command 

•  Deputy  Assistant  Secretary  of  the  Navy  (Acquisition  and  Logistics  Management) 

•  Office  of  Naval  Air  Systems  Command  PMA-290 

•  Office  of  the  Assistant  Secretary  of  the  Army  for  Acquisition,  Logistics  and 

Technology 

•  Director,  Strategic  Systems  Program 

•  Project  Manager  Modular  Brigade  Enhancements 

•  Deputy  Assistant  Secretary  Air  Force  (Management  Policy  &  Program  Integration) 

•  Dean  of  Research,  Naval  Postgraduate  School 

We  also  thank  the  Naval  Postgraduate  School  Foundation  and  acknowledge  its 
generous  contributions  in  support  of  this  Symposium. 

James  B.  Greene,  Jr.  Keith  F.  Snider,  PhD 

Rear  Admiral,  US  Navy  (Ret.)  Associate  Professor 


Karey  L.  Shaffer,  MBA 

Program  Manager,  Acquisition  Research  Program 
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Rear  Admiral  James  B.  Greene,  Jr.  USN  (Ret.) — Acquisition  Chair,  Naval 
Postgraduate  School.  RADM  Greene  develops,  implements  and  oversees  the  Acquisition 
Research  Program  in  the  Graduate  School  of  Business  and  Public  Policy.  He  interfaces  with 
DoD,  industry  and  government  leaders  in  acquisition,  coordinates  graduate  student  projects 
and  conducts  guest  lectures  and  seminars.  Before  serving  at  NPS,  RADM  Greene  was  an 
independent  consultant  focusing  on  Defense  Industry  business  development  strategy  and 
execution  (for  both  the  public  and  private  sectors),  minimizing  lifecycle  costs  through 
technology  applications,  alternative  financing  arrangements  for  capital-asset  procurement, 
and  “red-teaming”  corporate  proposals  for  major  government  procurements. 

RADM  Greene  served  as  the  Assistant  Deputy  Chief  of  Naval  Operations  (Logistics) 
in  the  Pentagon  from  1991-1995.  As  Assistant  Deputy,  he  provided  oversight,  direction  and 
budget  development  for  worldwide  US  Navy  logistics  operations.  He  facilitated  depot 
maintenance,  supply  chain  management,  base/station  management,  environmental 
programs  and  logistic  advice,  and  support  to  the  Chief  of  Naval  Operations.  Some  of  his 
focuses  during  this  time  were  leading  Navy-wide  efforts  to  digitize  all  technical  data  (and, 
therefore,  reduce  cycle-time)  and  to  develop  and  implement  strategy  for  procurement  of 
eleven  Sealift  ships  for  the  rapid  deployment  forces.  He  also  served  as  the  Senior  Military 
Assistant  to  the  Under  Secretary  of  Defense  (Acquisition)  from  1987-1990;  as  such,  he 
advised  and  counseled  the  Under  Secretary  in  directing  the  DoD  procurement  process. 

From  1984-1987,  RADM  Greene  was  the  Project  Manager  for  the  AEGIS  project. 
This  was  the  DoD’s  largest  acquisition  project,  with  an  annual  budget  in  excess  of  $5 
billion/year.  The  project  provided  oversight  and  management  of  research,  development, 
design,  production,  fleet  introduction  and  full  lifecycle  support  of  the  entire  fleet  of  AEGIS 
cruisers,  destroyers,  and  weapons  systems  through  more  than  2500  industry  contracts. 

From  1980-1984,  RADM  Greene  served  as  Director,  Committee  Liaison,  Office  of 
Legislative  Affairs  followed  by  a  tour  as  the  Executive  Assistant,  to  the  Assistant  Secretary 
of  the  Navy  (Shipbuilding  and  Logistics).  From  1964-1980,  RADM  Greene  served  as  a 
Surface  Warfare  Officer  in  various  duties,  culminating  in  Command-at-Sea.  His  assignments 
included  numerous  wartime  deployments  to  Vietnam,  as  well  as  the  Indian  Ocean  and  the 
Persian  Gulf. 

RADM  Greene  received  a  BS  in  Electrical  Engineering  from  Brown  University  in 
1964;  he  earned  an  MS  in  Electrical  Engineering  and  an  MS  in  Business  Administration  from 
the  Naval  Postgraduate  School  in  1973. 

Keith  F.  Snider — Associate  Professor  of  Public  Administration  and  Management  in 
the  Graduate  School  of  Business  &  Public  Policy  at  the  Naval  Postgraduate  School  in 
Monterey,  California,  where  he  teaches  courses  related  to  defense  acquisition  management. 
He  also  serves  as  Principal  Investigator  for  the  NPS  Acquisition  Research  Program  and  as 
Academic  Associate  for  resident  NPS  acquisition  curricula. 

Professor  Snider  has  a  PhD  in  Public  Administration  and  Public  Affairs  from  Virginia 
Polytechnic  Institute  and  State  University,  a  Master  of  Science  degree  in  Operations 
Research  from  the  Naval  Postgraduate  School,  and  a  Bachelor  of  Science  degree  from  the 
United  States  Military  Academy  at  West  Point.  He  served  as  a  field  artillery  officer  in  the  US 
Army  for  twenty  years,  retiring  at  the  rank  of  Lieutenant  Colonel.  He  is  a  former  member  of 
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the  Army  Acquisition  Corps  and  a  graduate  of  the  Program  Manager’s  Course  at  the 
Defense  Systems  Management  College. 

Professor  Snider’s  recent  publications  appear  in  American  Review  of  Public 
Administration,  Administration  and  Society,  Administrative  Theory  &  Praxis,  Journal  of  Public 
Procurement,  Acquisition  Review  Quarterly,  and  Project  Management  Journal. 

Karey  L.  Shaffer — Program  Manager  for  General  Dynamics  Information  Technology 
in  support  of  the  Acquisition  Research  Program  at  the  Graduate  School  of  Business  and 
Public  Policy,  Naval  Postgraduate  School.  As  PM,  Shaffer  is  responsible  for  operations  and 
publications  in  conjunction  with  the  Acquisition  Chair  and  the  Principal  Investigator.  She  has 
also  catalyzed,  organized  and  managed  the  Acquisition  Research  Symposiums  hosted  by 
NPS. 


Shaffer  has  also  served  as  the  Project  Manager  for  Imagicast,  Inc.,  and  as  the 
Operations  Manager  for  the  Montana  World  Trade  Center.  At  Imagicast,  she  was  asked  to 
take  over  the  project  management  of  four  failing  pilots  for  Levi  Strauss  in  the  San  Francisco 
office.  Within  four  months,  the  pilots  were  released;  the  project  lifecycle  was  shortened;  and 
the  production  process  was  refined.  In  this  latter  capacity  at  the  MWTC,  Shaffer  developed 
operating  procedures,  policies  and  processes  in  compliance  with  state  and  federal  grant 
law.  Concurrently,  she  managed  $1.25  million  in  federal  appropriations,  developed 
budgeting  systems  and  secured  a  $400,000  federal  technology  grant.  As  the  Operations 
Manager,  she  also  designed  MWTC’s  Conference  site,  managed  various  marketing 
conferences,  and  taught  student  practicum  programs  and  seminars. 

Shaffer  holds  an  MBA  from  San  Francisco  State  University  and  earned  her  BA  in 
Business  Administration  (focus  on  International  Business,  Marketing  and  Management)  from 
the  University  of  Montana. 

A  special  thanks  to  our  editors  Jeri  Larsen,  Breanne  Grover  and  Jessica  Moon  for 
all  that  they  have  done  to  make  this  publication  a  success,  to  David  Wood,  Tera  Yoder  and 
Ian  White  for  production  and  graphic  support  and  to  the  staff  at  the  Graduate  School  of 
Business  &  Public  Policy  for  their  administrative  support.  Our  program  success  is  directly 
related  to  the  combined  efforts  of  many. 
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ACQUISITION  RESEARCH: 

CREATING  SYNERGY  FOR  INFORMED  CHANGE 

6th  Annual  Acquisition  Research  Symposium 

May  13  -  14,  2009 
Monterey,  California 


Announcement  and  Call  for  Proposals 

The  Graduate  School  of  Business  &  Public  Policy  at  the  Naval  Postgraduate  School 
announces  the  6th  Annual  Acquisition  Research  Symposium  to  be  held  May  13-14,  2009 
in  Monterey,  California. 

This  symposium  serves  as  a  forum  for  the  presentation  of  acquisition  research  and  the 
exchange  of  ideas  among  scholars  and  practitioners  of  public-sector  acquisition.  We  seek  a 
diverse  audience  of  influential  attendees  from  academe,  government,  and  industry  who  are 
well  placed  to  shape  and  promote  future  research  in  acquisition. 

The  Symposium  Program  Committee  solicits  proposals  for  panels  and/or  papers  from 
academicians,  practitioners,  students  and  others  with  interests  in  the  study  of  acquisition. 
The  following  list  of  topics  is  provided  to  indicate  the  range  of  potential  research  areas  of 
interest  for  this  symposium:  acquisition  and  procurement  policy,  supply  chain 
management,  public  budgeting  and  finance,  cost  management,  project  management, 
logistics  management,  engineering  management,  outsourcing,  performance 
measurement,  and  organization  studies. 

Proposals  must  be  submitted  by  November  7,  2008.  The  Program  Committee  will  make 
notifications  of  accepted  proposals  by  December  5,  2008.  Final  papers  must  be  submitted 
by  April  3,  2009  to  be  included  in  the  Symposium  Proceedings. 

Proposals  for  papers  should  include  an  abstract  along  with  identification,  affiliation,  and 
contact  information  for  the  author(s).  Proposals  for  papers  plan  for  a  20  minute 
presentation.  Proposals  for  panels  (plan  for  90  minute  duration)  should  include  the  same 
information  as  above  as  well  as  a  description  of  the  panel  subject  and  format,  along  with 
participants’  names,  qualifications  and  the  specific  contributions  each  participant  will  make 
to  the  panel. 


Submit  paper  and  panel  proposals  to  www.researchsvmposium.org  . 
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Keynote  Speaker 


Thursday, 

May  15,  2008 

Keynote  Speaker 

8:00  a.m.  - 

Keynote  Speaker 

9:15  a.m. 

General  Bruce  Carlson,  United  States  Air  Force,  Commander,  Air  Force 
Materiel  Command 

General  Bruce  Carlson  serves  as  Commander,  Air  Force 
Materiel  Command,  Wright-Patterson  Air  Force  Base,  OH. 

The  command  conducts  research,  development,  test  and 
evaluation,  and  provides  acquisition  management  services 
and  logistics  support  necessary  to  keep  Air  Force  weapon 
systems  ready  for  war. 

Carlson  was  born  in  Hibbing,  MN.  He  was  commissioned  in 
1971 ,  after  completing  the  University  of  Minnesota's  Air  Force 
ROTC  program  as  a  distinguished  graduate.  He  has  held 
various  assignments  in  flying  units.  Staff  assignments  have 
included  positions  at  Tactical  Air  Command,  Headquarters 
US  Air  Force,  the  offices  of  the  Secretary  of  the  Air  Force  and 
Secretary  of  Defense,  and  as  the  Director  of  Force  Structure, 

Resources  and  Assessment  with  the  Joint  Staff.  Additionally, 
he  commanded  the  Air  Force's  stealth  fighter  wing,  the  49th, 
at  Holloman  AFB,  NM.  Prior  to  assuming  his  current  position, 

Carlson  served  as  the  Commander,  8th  Air  Force,  Barksdale 
AFB,  LA,  and  Joint  Functional  Component  Commander  for 
Space  and  Global  Strike,  US  Strategic  Command,  Offutt 
AFB,  NE. 

Carlson  is  experienced  in  multiple  aircraft  weapons  systems,  is  a  command  pilot  with  more  than 
3,000  flying  hours,  and  has  combat  experience  in  the  OV-10. 
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Panel  1 1  -  Plenary  Panel  -  Academic  Research  on 
Contracting 


Thursday, 
May  15,  2008 


9:30  a.m.  -  Chair: 
11:00.m. 


Dr.  Steve  Kelman,  Professor  of  Public  Management,  John  F.  Kennedy 
School  of  Government,  Harvard  University 

Discussants: 

Dr.  Matthew  Potoski,  Associate  Professor,  Department  of  Political 
Science,  Iowa  State  University 

Dr.  G.  Frederick  Thompson,  Professor  of  Public  Management  and  Policy, 
Atkinson  Graduate  School  of  Management,  Willamette  University 

Dr.  Andrew  B.  Whitford,  Associate  Professor  of  Public  Administration  and 
Policy,  School  of  Public  &  International  Affairs,  The  University  of  Georgia 


Chair:  Dr.  Steve  Kelman  is  the  Weatherhead  Professor  of  Public  Management  at  Harvard 
University’s  John  F.  Kennedy  School  of  Government.  A  summa  cum  laude  graduate  of  Harvard 
College,  with  a  PhD  in  Government  from  Harvard  University,  he  is  the  author  of  many  books  and 
articles  on  the  policymaking  process  and  on  improving  the  management  of  government  organizations. 
His  most  recently  published  books  are  a  study  on  how  to  improve  the  government  computer 
procurement  process,  entitled  Procurement  and  Public  Management:  The  Fear  of  Discretion  and  the 
Quality  of  Government  Performance  (AEI  Press,  1 990),  and  Making  Public  Policy:  A  Hopeful  View  of 
American  Government  (Basic  Books,  1987).  In  1996,  he  was  elected  a  Fellow  of  the  National 
Academy  of  Public  Administration.  In  2001,  he  received  the  Herbert  Roback  Memorial  Award,  the 
highest  achievement  award  of  the  National  Contract  Management  Association.  In  2003,  he  was 
elected  as  a  Director  of  The  Procurement  Roundtable. 

From  1993  through  1997,  Kelman  served  as  Administrator  of  the  Office  of  Federal  Procurement 
Policy  in  the  Office  of  Management  and  Budget.  During  his  tenure  as  Administrator,  he  played  a  lead 
role  in  the  Administration’s  “reinventing  government”  effort.  He  led  Administration  efforts  in  support  of 
the  Federal  Acquisition  Streamlining  Act  of  1 994  and  the  Federal  Acquisition  Reform  Act  of  1 995. 

Discussant:  Dr.  Matthew  Potoski  is  an  Associate  Professor  at  Iowa  State  University,  where  he 
teaches  courses  on  politics,  administration,  and  policy.  In  1997  and  1998,  he  taught  at  the  University 
of  Kentucky’s  Martin  School  of  Public  Policy  and  Administration,  first  as  a  visiting  fellow  and  later  as 
visiting  faculty.  Potoski  received  his  PhD  from  Indiana  University  in  1998,  his  undergraduate  degree 
from  Franklin  and  Marshall  College  in  Lancaster,  PA,  and  his  Master’s  degree  from  the  University  of 
Vermont. 

Potoski’s  research  and  teaching  interests  lie  at  the  nexus  of  public  policy,  administration,  and  politics. 
His  research  examines  how  people  address  collective  action  problems  and  uncertainty  inherent  in 
developing,  implementing  and  managing  public  policies.  These  problems  in  various  guises  undermine 
peoples’  ability  to  solve  complex  problems  and  produce  sound  public  policy.  Such  problems  span 
levels  of  government  and  policy  areas.  He  has  published  over  two  dozen  articles  in  journals  such  as 
American  Journal  of  Political  Science,  Journal  of  Politics,  Journal  of  Policy  Analysis  and 
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Management,  Public  Administration  Review,  Policy  Studies  Journal,  State  Politics  and  Policy 
Quarterly  and  the  Journal  of  Public  Administration  Research  and  Theory.  Potoski  is  the  recipient  of 
the  Indiana  University  Department  of  Political  Science’s  2000  Greenough  Award  for  best  dissertation 
completed  in  previous  year,  and  the  Iowa  State  University  College  of  Liberal  Arts  and  Sciences’  2004 
Award  for  Early  Achievement  in  Research/Artistic  Creativity.  He  is  the  co-editor  of  International  Public 
Management  Journal  and  serves  on  the  editorial  board  of  Journal  of  Public  Administration  Research 
and  Theory. 

Discussant:  Dr.  G.  Frederick  Thompson  is  the  Grace  and  Elmer  Goudy  Professor  of  public 
management  and  policy  at  Atkinson  Graduate  School  of  Management,  Willamette  University. 

Discussant:  Dr.  Andrew  B.  Whitford  is  an  Associate  Professor  of  public  administration  and  policy  in 
the  University  of  Georgia  School  of  Public  and  International  Affairs.  He  is  a  specialist  in  the  areas  of 
bureaucratic  politics,  organization  theory  and  political  economy.  His  interests  in  public  policy  include 
environmental,  regulation  and  public  health  policy,  and  his  research  has  been  supported  by  the 
National  Science  Foundation  (NSF),  the  Robert  Wood  Johnson  Foundation  and  the  Russell  Sage 
Foundation.  He  has  served  on  NSF  advisory  panels  and  is  on  the  editorial  board  of  Political  Research 
Quarterly.  His  articles  have  been  published  in  Administrative  Science  Quarterly,  American  Journal  of 
Political  Science,  British  Journal  of  Political  Science  and  the  Journal  of  Public  Administration 
Research,  and  he  is  working  on  a  book  manuscript  about  presidential  rhetoric  in  the  war  on  drugs. 
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Panel  12  -  Budgeting  and  the  Acquisition  Process 


Thursday, 
May  15,  2008 


11:15  a.m.-  Chair: 
12:45  a.m. 


The  Honorable  Douglas  A.  Brook,  Assistant  Secretary  of  the  Navy  for 
Financial  Management  and  Comptroller 

Discussant: 

Ms.  Wendy  Kune,  Director,  Naval  Center  for  Cost  Analysis 

Rear  Admiral  Charles  H.  Goddard,  Program  Executive  Officer,  Ships 


Paper: 


Rethinking  Acquisition  Reform:  Cost  Growth  Solutions  May  Aggravate  the 
More  Important  Problem 


Philip  J.  Candreva,  Senior  Lecturer  of  Budgeting,  Naval  Postgraduate 
School 


Chair:  The  Honorable  Douglas  A.  Brook  is  Assistant  Secretary  of  the  Navy  (Financial 
Management  &  Comptroller).  Prior  to  reporting  to  the  Pentagon,  he  was  Professor  of  Public  Policy 
and  Director  of  the  Center  for  Defense  Management  Reform  at  the  Naval  Postgraduate  School  in 
Monterey,  California.  From  2002  until  2005,  Dr.  Brook  was  Dean  of  the  NPS  Graduate  School  of 
Business  and  Public  Policy. 

Before  joining  NPS,  Dr.  Brook  was  Vice  President — Government  Affairs  for  The  LTV  Corporation. 
Prior  to  joining  LTV,  Brook  served  in  two  Presidentially  appointed  positions.  In  1 992,  he  was  Acting 
Director  of  the  US  Office  of  Personnel  Management  (OPM),  the  central  personnel  management 
agency  of  the  Federal  government.  From  1 990  to  1 992,  Brook  was  Assistant  Secretary  of  the  Army 
for  Financial  Management. 

Brook  began  his  career  as  Director  of  Public  Finance  of  the  National  Association  of  Manufacturers  in 
New  York.  Subsequently,  he  joined  the  Libbey-Owens-Ford  Company  and  served  as  Vice  President 
and  head  of  the  company’s  Washington,  DC,  office. 

In  1982,  he  founded  Brook  Associates,  Inc.,  a  public  affairs  consulting  business  serving  corporate 
and  trade  association  clients,  which  he  managed  until  assuming  duties  at  the  Pentagon.  He  also 
served  two  elected  terms  on  the  Town  Council  of  Vienna,  VA. 

Brook  grew  up  in  East  Detroit,  Ml.  He  attended  the  University  of  Michigan,  graduating  with  a  Bachelor 
of  Arts  degree  in  Political  Science  in  1965  and  a  Master  of  Public  Administration  degree  in  1967.  In 
2001,  he  earned  his  PhD  in  Public  Policy  at  George  Mason  University.  He  also  completed  the  1977 
Executive  Program  at  the  University  of  Virginia’s  Colgate  Darden  Graduate  School  of  Business 
Administration. 
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Brook  served  on  active  duty  as  a  Navy  Supply  Corps  officer  and  was  a  member  of  the  Naval  Reserve 
for  30  years.  He  retired  with  the  rank  of  Captain. 

Discussant:  Ms.  Wendy  P.  Kune  was  selected  to  the  Senior  Executive  Service  in  March  2005  and 
currently  serves  as  Director,  Naval  Center  for  Cost  Analysis  (NCCA)  and  leader  of  the  Department  of 
the  Navy  Cost  Analysis  and  Management  Sciences  community.  Prior  to  holding  this  position,  Kune 
headed  NCCA’s  Cost  Analysis  Tools  division  and  served  as  Naval  VAMOSC  program  manager. 

Kune  held  several  positions  within  the  Department  of  the  Air  Force,  including  Chief  of  the  Cost 
Factors  Branch  within  the  Air  Force  Cost  Analysis  Agency.  She  also  led  the  transition  of  Air  Force 
VAMOSC  to  the  more  comprehensive  Air  Force  Total  Ownership  Cost  (AFTOC)  system. 

Kune  holds  a  Bachelor’s  degree  in  Mathematics  from  the  University  of  Missouri  and  Master  of 
Science  degrees  in  Computer  Information  Systems  from  St.  Mary’s  University,  San  Antonio,  and 
National  Resource  Strategy  from  the  National  Defense  University.  She  completed  the  National 
Defense  University’s  CIO  certification  program  in  2005.  Kune  is  a  Certified  Defense  Financial 
Manager,  is  Level  III  certified  in  the  Defense  Acquisition  Workforce,  and  is  a  member  of  the 
Acquisition  Corps. 

Discussant:  Rear  Admiral  Charles  Goddard  graduated  from  the  US  Naval  Academy  in  1978  with  a 
Bachelor  of  Science  degree  in  Naval  Architecture.  He  also  holds  a  Master  of  Science  degree  in  Naval 
Architecture  and  Ocean  Engineer  from  the  Massachusetts  Institute  of  Technology  (MIT). 

Goddard  achieved  Surface  Warfare  Qualification  in  USS  Robert  E.  Perry  (FF  1073),  where  he  served 
as  Anti-Submarine  Warfare  Officer  and  Auxiliaries  and  Electrical  Officer. 

He  became  an  Engineering  Duty  Officer  in  1981.  His  engineering  duty  officer  tours  include:  Pearl 
Harbor  Naval  Shipyard  as  Ship  Superintendent,  1981-1982;  Long  Beach  Naval  Shipyard  as  Ship 
Superintendent  and  Type  Desk  Officer,  1986-1989;  David  Taylor  Research  Center  as  System 
Engineer  for  the  Advanced  Surface  Machinery  Program,  1989-1992;  Naval  Sea  Systems  Command 
as  Surface  Stealth  Project  Officer,  1992-1995;  Supervisor  of  Shipbuilding  San  Diego  as  New 
Construction  Officer,  1995-1998;  Naval  Sea  Systems  Command  as  Director  of  Ship  Research  and 
Development,  1998;  and  as  Strategic  Sealift  Deputy  Program  Manager  (PMS  385),  1999.  Goddard 
served  as  a  Chief  of  Naval  Operations  Fellow  on  the  Strategic  Studies  Group  from  1999-2000, 
Executive  Assistant  to  Commander,  Naval  Sea  Systems  Command  from  2000-2001 ;  DD(X)  Program 
Manager  from  2001-2005;  and  as  Vice  Chief,  Naval  Sea  System  Command  from  2005  to  2007.  He 
assumed  his  present  duties  in  February  2007.  He  is  a  member  of  the  Acquisition  Professional 
Community  and  a  graduate  of  the  Program  Manager’s  Course  at  the  Defense  Systems  Management 
College. 

Goddard’s  personal  decorations  include  the  Legion  of  Merit  (two  awards),  Navy  Meritorious  Service 
Medal  (two  awards)  and  Navy  Commendation  Medal  (two  awards).  He  is  the  author  of  several  articles 
on  the  topic  of  ship  design  and  construction,  and  recipient  of  the  American  society  of  Naval  Engineers 
Gold  Medal  Award  for  his  contributions  to  ship  design  and  engineering. 
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Rethinking  Acquisition  Reform:  Cost-growth  Solutions  May 
Aggravate  More  Important  Problems 


Presenter:  Phil  Candreva,  M.S.,  Senior  Lecturer  of  Budgeting,  Graduate  School  of  Business 
and  Public  Policy,  NPS.  Candreva’s  research  investigates  how  government  organizations  use 
financial  information  in  such  areas  as  resource  allocation  decision-making,  accounting, 
performance  measurement,  and  management  reform.  Most  contemporary  public  sector 
management  reform  efforts  are  either  explicitly  tied  to  financial  decisions  (e.g.,  performance- 
based  budgeting)  or  are  implicitly  tied  through  other  management  efforts  (e.g.,  efficiency 
programs).  Since  budgets  are  the  battlefield  on  which  public  policy  disputes  are  waged,  public 
managers  must  become  proficient  at  showing  how  effectively  and  efficiently  those  resources  are 
being  used  in  order  to  preserve  or  expand  their  resource  base.  Such  efforts  are  a  critical 
dimension  of  contemporary  management  reform. 

Philip  J.  Candreva 
Senior  Lecturer  of  Budgeting 

Senior  Associate,  Center  for  Defense  Management  Reform 

Naval  Postgraduate  School 

555  Dyer  Road 

Monterey,  CA  93943 

Phone:  831-656-2884 

E-mail:  pjcandre@nps.ed 


Abstract 

There  is  increasing  dissatisfaction  with  cost  growth  in  major  defense  acquisition 
programs.  Cost  growth  crowds  out  other  investments,  stresses  budgets  or  causes  schedule 
slips,  all  of  which  result  in  a  military  force  that  is  less  capable  than  previously  expected. 
Several  recent  studies  have  recommended  two  categories  of  reforms:  capital  budgeting 
reforms  seek  stability  in  acquisition  accounts,  and  rational  cost  model  reforms  seek  to 
reduce  the  percentage  increase  of  final  cost  over  budget  estimates.  In  both  categories, 
undesirable  secondary  effects  may  be  worse  than  the  desirable  primary  effects;  specifically, 
reforms  that  reduce  cost  growth  may  do  so  by  driving  total  costs  higher.  This  study 
examines  these  reforms  and  discusses  their  secondary  effects.  The  paper  concludes  that 
the  current  practice  of  generating  low  estimates,  coupled  with  dissatisfaction  with  cost 
growth  may  best  serve  to  limit  total  cost. 

Introduction 

Cost  growth  in  defense  acquisition  programs  is  a  problem  that  captures  the  attention 
of  the  general  public,  the  media,  Congress,  defense  reformers,  and  the  acquisition 
community.  Cost  growth  is  said  to  reduce  the  affordability  of  the  long-term  defense  program, 
resulting  in  top-line  budget  increases,  reduced  or  late  programs,  or  a  combination  of  the  two. 
Despite  over  a  decade  of  growing  defense  budgets  (even  when  one  excludes  the  burden 
and  cost  of  GWOT  borne  outside  the  regular  budget  process),  service  chiefs  feel 
increasingly  constricted  in  their  procurement  accounts.  From  2001  to  2007,  the  planned 
investment  in  new  programs  doubled — from  a  portfolio  valued  at  approximately  $750  billion 
to  almost  $1.5  trillion  (GAO,  2007,  March,  p.  3).  Cost  growth  is  a  persistent  and  perhaps 
intractable  problem: 
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■  Historical  performance  has  not  been  good.  The  GAO  has  included  defense 
acquisition  management  among  its  High  Risk  programs  since  the  list  began  in  1990, 
in  part  due  to  cost  management.  “Weapons  systems  routinely  take  much  longer  to 
field,  cost  more  to  buy,  and  require  more  support  than  provided  for  in  investment 
plans”  (GAO,  2007,  January,  p.  61).  The  GAO’s  review  of  27  weapons  systems 
showed  total  cost  growth  of  19%,  RDT&E  growth  of  over  33%  and  schedule 
slippages  of  23%  (GAO  2007,  March,  p.  9).  Looking  back  over  the  last  three 
decades,  RAND  has  shown  that  cost  growth  has  averaged  about  46% — that  is, 
programs  actually  cost  46%  more  than  estimated  at  Milestone  B — and  that  cost 
performance  has  not  improved  despite  numerous  and  continuous  acquisition  reform 
efforts  (Younossi  et  al.,  2007). 

■  The  problem  is  expected  to  continue  into  the  future.  Using  historical  performance  as 
a  predictor  of  future  needs,  the  CBO  projects  that  the  investment  accounts  in  the 
FYDP  are  underfunded  by  about  28%  (CBO,  2007,  p.  14). 

■  The  phenomena  are  well-understood  and  documented.  The  Defense  Acquisition 
Performance  Assessment  {DAP A)  report  stated: 


Over  many  years,  128  studies  have  been  done  to  address  perceived 
problems  with  the  system  and  to  prevent  waste,  fraud  and  abuse.  Historically, 
we  observed  that  cost  and  schedule  instability  have  been  a  problem  in  all 
system  acquisitions  since  the  Civil  War.  We  see  some  of  the  same  issues  as 
problems  today  that  the  Packard  Commission  saw  20  years  ago.  (Kadish  et 
al.,  2006,  p.  2) 


■  The  critics  and  ideas  are  not  all  external  to  the  Department.  The  DoD’s  own 
Acquisition  Transformation  Report  to  Congress  lists  seven  implementation  goals. 
Three  include  the  term  “cost-effective”  in  their  title,  and  a  fourth  relates  to 
governance  and  decision  processes  (Kreig,  2007,  p.  6).  It  is  not  that  DoD  is  turning  a 
deaf  ear;  the  intentions  are  all  good. 

There  is  no  shortage  of  recommendations  and  plans  to  fix  the  problems  of  cost 
growth.  Defense  contractors,  advisory  bodies,  and  R&D  centers  have  spent  vast  sums  over 
the  years  chronicling  such  problems  and  offering  solutions.  Ninety  (42%)  of  the  212  defense 
management  reform  initiatives  in  the  period  1990-2006  dealt  with  acquisition  and  budgeting 
(Francis  &  Walther,  2006).  The  study  of  reform  agendas  and  recommendations  is  an 
exercise  in  organizational  evolution.  Just  as  Darwin  noted  the  beaks  of  the  finches  on  the 
Galapagos  Islands  adapted  gradually  to  their  environments,  recommendations  to  fix 
acquisition  evolve  slightly  from  report  to  report.  This  article  takes  a  radically  different  tack — a 
leap  on  the  reform  recommendation  evolutionary  path,  if  you  will.  This  article  will  argue  that 
the  cost-growth  phenomenon  may  not  even  be  a  problem.  It  probably  is  less  of  a  problem 
than  the  proffered  solutions.  In  fact,  if  we  assume  DoD  policies  and  procedures  have 
evolved  purposefully  with  competent  managers  in  light  of  the  available  knowledge,  then  the 
current  system  may  be  the  best  available  to  ensure  programs  are  delivered  at  the  lowest 
cost.  Attributes  of  some  of  the  solutions  to  cost  growth  may  actually  drive  total  costs  higher. 

Those  who  study  defense  acquisition  point  to  several  problems  associated  with  cost 
growth.  One  set  of  problems  is  related  to  the  decision  whether  to  invest.  Underestimation 
“leads  to  poor  investment  choices”  by  starting  more  or  larger  programs  than  the  department 
can  afford  (Melese,  Franck,  Angelis  &  Dillard,  2007,  p.  358).  If  there  is  a  business  rule  that  a 
certain  cost-benefit  ratio  threshold  must  be  exceeded  to  make  the  initial  investment,  then  an 
underestimate  of  cost  may  cause  those  making  resource  allocation  decisions  to  err. 


ACQUISITION  RESEARCH:  CREATING  SYNERGY  FOR  INFORMED  CHANGE  -  254  - 


Similarly,  in  an  analysis  of  alternatives,  if  the  cost  estimates  are  not  uniformly  inaccurate, 
resource-allocation  decision-makers  may  make  the  wrong  choice. 

Another  set  of  problems  is  related  to  the  effects  of  growth  after  it  occurs.  Because 
the  budget  is  a  social  contract,  breaches  of  cost  estimates  can  damage  trust  and 
relationships  within  government  (Melese  et  al.,  2007,  p.  359).  In  some  cases,  they  may 
breach  legally  imposed  thresholds,  the  so-called  Nunn-McCurdy  breaches.  When  costs 
overrun,  the  remedy  may  involve  stretching  out  schedules  or  reducing  quantity,  thereby 
reducing  the  new  system’s  effect  on  the  operating  forces.  The  remedy  may  instead  be  a 
reallocation  of  funds  away  from  a  lower-priority  program,  thereby  causing  a  different 
program’s  schedule  or  quantity  to  suffer.  The  remedy  may  also  be  a  request  for  additional 
budget  authority,  imposing  an  opportunity  cost  for  the  nation  as  a  whole. 

The  two  most  commonly  proposed  solutions  to  cost  growth  address  the  two  sides  of 
the  problem  that  are  most  under  governmental  control:  budget  stability  and  cost-estimation 
accuracy.  The  former  contends  that  increased  funding  stability  would  allow  the  program 
manager  to  shift  attention  to  non-financial  factors  by  insulating  the  program  from  the 
vagaries  of  politics  and  execution-year  fiscal  maneuvers.  The  GAO  reported  that  over  one- 
third  of  program  managers  said  the  biggest  obstacle  they  faced  was  funding  stability  (GAO, 
2005,  p.  44).  The  latter  contends  that  the  problem  originates  with  inaccurate  cost  estimates 
and  that  improved  accuracy  will  contain  cost  growth.  These  reforms  address  the  problem  of 
unmet  expectations  by  minimizing  the  amount  of  cost  growth.  In  the  process  of  doing  so, 
they  unfortunately  fail  to  alleviate  the  effects  of  cost  growth.  In  some  cases,  these  reforms 
would  introduce  additional  undesirable  effects.  This  paper  argues  that  instead  of  seeking  to 
minimize  relative  cost  growth,  decision-makers  should  focus  attention  on  minimizing  total 
cost.  Paradoxically,  some  of  the  very  forces  that  hold  total  cost  down  are  rooted  in  the 
dissatisfaction  with  cost  growth. 

Capital  Budgeting  Reforms 

Private  industry — as  well  as  many  state  and  municipal  governments — employs 
separate  processes  to  budget  for  capital  items  and  operating  expenses.  Most  people  in 
managing  their  household  budgets  do  the  same:  we  apply  one  form  of  decision  logic  when 
budgeting  for  the  electric  bill  or  groceries  and  a  different  logic  when  deciding  to  purchase  a 
new  car  or  major  appliance.  Within  the  DoD,  procurement  budgets  are  worked  alongside 
operating  and  salary  budgets,  and  both  are  appropriated  on  an  annual  line-item  basis. 
Reformers  have  often  questioned  this  practice.  Most  recently,  the  2006  Quadrennial 
Defense  Review  and  the  Defense  Acquisition  Performance  Assessment  recommended  a 
form  of  capital  budgeting. 

Capital  budgeting  involves  the  analysis  of  costs  and  cash  flows  associated  with  an 
investment  project  that  precedes  and  informs  the  decision  to  invest.  At  most  levels  of 
government  (excepting,  notably,  the  federal  government),  capital  investment  decisions  are 
made  in  tandem  with  the  decision  of  how  to  finance  them — most  being  paid  for  with  revenue 
measures  such  as  a  special  tax  assessment  or  with  debt  instruments  such  as  bond 
issuances  or  loans.  There  is  a  deliberate  link  between  the  investment  and  its  implications  for 
current  and  long-term  budgets,  fiscal  policy,  asset  management,  and  cash  flows  (Lee  & 
Johnson,  1998). 

The  budgeting  practice  for  capital  investments  at  the  federal  level  has  a  different 
flavor.  Federal  capital  budgets  do  not  necessarily  need  to  be  separate  from  operating 
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budgets  in  the  absence  of  the  balanced  budget  requirements  common  at  the  state  and 
municipal  level.  The  sovereign  federal  government  may  run  deficits,  raise  revenues,  print 
money,  and  borrow  more  readily  than  any  other  organization.  Linking  the  capital  purchase  to 
a  specific  revenue  stream  is  not  necessary.  Annual  resource  flows  are  often  sufficient  to  fully 
fund  the  federal  capital  projects.  Thus,  they  face  they  same  annual  review  and  trade-off 
decision-making  as  operating  budget  accounts. 

Irrespective  of  the  budgetary  concern,  federal  capital  projects  are  sometimes  viewed 
as  mechanisms  for  affecting  macroeconomic  conditions.  Capital  projects  may  be  started,  not 
because  the  ends  of  the  project  are  the  main  objective,  but  because  the  job  itself,  the  means 
to  the  end,  is  highly  desirable.  Programs  during  the  Great  Depression  like  the  PWA,  WPA 
and  TVA  were  as  much  about  creating  jobs  as  they  were  about  building  bridges  and  dams. 
Urban  renewal  projects  are  more  about  sociological  factors  than  buildings.  Likewise,  some 
capital  investments  in  military  systems  are  made  in  part  to  effect  public  objectives  other  than 
military  capability.  These  short-term  and  complementary  goals  that  are  achieved  through 
capital  programs  confound  the  analysis  and  argue  against  a  separate  capital  budgeting 
strategy. 

Private-sector  capital  budgeting  practices  are  similar  to  those  used  in  government 
and  include  lifecycle  cost  and  benefit  analyses.  The  costs  include  the  obvious  investment  in 
the  item,  financing  costs,  and  any  incremental  operating  and  support  costs.  Benefits  include 
new  revenue  streams  or  lower  operating  and  support  costs.  Investment  decisions  are  based 
on  one  or  more  analytical  techniques,  such  as  payback  period,  net  present  value,  or  internal 
rate  of  return.  These  are  benchmarked  against  a  hurdle  rate  which  represents  the  next  best 
available  use  of  the  funds.  Assuming  the  project  will  generate  sufficient  risk-adjusted  return 
in  a  reasonable  period  of  time,  a  capital  budget  is  prepared.  That  budget  is  often  distinct 
from,  but  affects,  the  organization’s  annual  operating  and  support  budget.  The  capital 
budget  may  be  prepared  outside  the  annual  operating  and  support  budget  cycle,  the  timing 
more  aligned  with  project  schedules  than  accounting  cycles.  Funds  to  pay  for  the  capital 
project  also  may  be  raised  separately  from  the  revenues  raised  by  routine  operations  (but 
the  service  of  debt,  for  example,  would  be  incorporated  into  future  operating  budgets). 

One  significant  problem  with  adapting  capital  budgeting  for  defense  items  is  that 
there  is  no  benefit  that  is  easily  defined  in  financial  terms.  Computation  of  a  net  present 
value  or  payback  period  is  meaningless  if  the  benefit  and  the  cost  cannot  be  expressed  in 
consistent  units.4  There  is  no  clear  rate  of  return  against  which  to  compare  to  a  hurdle  rate. 
What  can  be  done,  however,  is  to  perform  a  cost-effectiveness  analysis  for  competing 
proposals  that  perform  essentially  the  same  task.  Two  alternative  proposals  for  meeting  the 
same  need  have  presumably  equal  benefits  and,  therefore,  can  be  compared  based  solely 
on  a  cost  analysis.  But  comparison  of  capital  budgets  for  items  that  generate  disparate 
benefits  is  very  problematic. 

Acquisition  reformers  argue  that  best  (or  at  least  common)  capital  budgeting 
practices  would  provide  much  needed  stability  to  the  management  of  defense  acquisition 
programs  (Kadish  et  al.,  2006;  McCaffery  &  Jones,  2006).  One  form  of  the  proposal  is  to 
separate  the  capital  decision  from  the  operating  cost  decision,  to  examine  each  capital 
decision  when  the  program  is  ready,  and  to  fund  fully  the  development  and  acquisition  costs 


4  Housel  and  Bell  (2001)  recognize  this  problem  and  suggest  a  methodology.  Practical  application  of 
that  methodology  in  the  DoD  is  unlikely  in  the  near  term,  if  ever. 
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at  that  time.  It  is  argued  that  such  a  practice  will  increase  the  likelihood  of  program  success 
by  addressing  a  significant  risk  factor.  The  FY  2007  Defense  Acquisition  Transformation 
Report  to  Congress  clearly  shows  that  the  DoD  accepts  this  argument  (Kreig,  2007,  p.  20). 
Such  proposals,  however,  involve  risk.  Indeed,  secondary  costs  may  outweigh  the  primary 
benefit. 

Isolating  the  capital  investment  decision  necessarily  removes  it  from  the  larger 
discussion  of  overall  defense  policy  and  resource  allocation.  The  legislature  prefers  to 
debate  and  decide  defense  matters  once  in  the  annual  authorization  and  appropriation 
process.  Separating  the  capital  items  may  cause  the  same  issues  to  be  debated  twice  or 
may  mask  interrelationships  that  should  be  considered  as  a  whole.  Just  as  the  DoD 
considers  the  full  portfolio  of  options  during  the  programming  phase  of  PPBE,  the  legislature 
needs  to  do  the  same.  By  separating  the  capital  items  from  the  rest  of  the  program  during 
budgeting,  decision-makers  could  eliminate  that  possibility.  Given  the  legislature’s 
constitutional  power  to  raise  and  support  armies  and  provide  and  maintain  a  navy,  such 
proposals  seem  politically  infeasible. 

Separation  of  the  capital  investment  decision  also  has  little  practical  benefit.  Under 
existing  full-funding  policies,  the  entire  amount  to  build  an  end-item  is  presumably  budgeted 
and  appropriated.  If  prepared  properly,  the  budget  request  should  have  considered  the 
lifecycle  cost  and  independent  cost  analyses.  If  Congress  appropriates  the  full  funding  in 
multi-year  appropriations,  based  on  the  President’s  Budget,  then  the  goals  of  a  capital 
budgeting  process  have  already  been  met.  Curiously,  the  trend  in  the  DoD,  at  least  in  the 
Navy,  has  been  to  expand  the  use  of  incremental  budgeting  over  full  funding — a  practice 
clearly  at  odds  with  the  stated  recommendation  to  adopt  corporate-style  capital  budgeting 
practices. 

Another  form  of  capital  budgeting  reform  proposes  to  fence  procurement  dollars 
once  appropriated.  The  problem  this  proposal  attempts  to  remedy  is  that  funds  too  easily 
move  out  of  procurement  programs  to  address  contingent  needs  elsewhere  in  the  budget.  In 
some  cases,  one  procurement  program  which  is  experiencing  problems  is  assisted  with 
funds  taken  from  another  procurement  program.  In  other  cases,  contingencies  may  affect 
the  operating  accounts,  and  without  sufficient  budgetary  slack  to  address  the  contingency, 
funds  are  transferred  from  procurement  programs.  The  recommendation  to  fence  funds, 
however,  is  a  budgeting  reform  to  what  is  actually  a  problem  of  execution  discipline.  If  the 
services  are  concerned  that  funds  are  taken  from  one  program  to  address  issues  in  another, 
then  they  should  simply  stop  doing  that.  Addressing  the  problem  of  execution  discipline  is  a 
more  appropriate  remedy  than  creating  a  new  budgetary  approach.  Easier  said  than  done, 
critics  will  say.  So  if  this  reallocation  is  inevitable,  what  are  the  effects  of  such  fences? 

Fenced  accounts  serve  to  make  the  resources  allocated  to  procurement  more 
important  than  dollars  allocated  to  operations;  or,  to  put  it  another  way,  future  readiness 
would  become  de  facto  more  important  than  current  readiness.  It  would  be  short-sighted  to 
institutionalize  such  a  decision,  as  it  actually  restricts  flexibility  to  deal  with  unforeseen 
contingencies.  If  the  procurement  accounts  cannot  be  a  source  to  deal  with  the  contingency, 
what  is  the  source?  Pay  accounts?  Operations  elsewhere  around  the  globe?  Such  fences 
also  tend  to  restrict  funds  flowing  in  both  directions.  Many  an  acquisition  program  has 
benefited  from  reprogramming  actions  in  which  funds  are  taken  from  operating  accounts; 
separate  capital  accounts  may  restrict  these  flows.  The  Mine  Resistant  Ambush  Protected 
(MRAP)  vehicle  program  received  over  a  billion  dollars  in  FY  2007  that  way.  The  current 
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process  is  not  perfect,  but  it  may  be  better  than  inappropriately  adopting  a  practice  that  is 
“best”  in  a  different  context. 

To  summarize,  corporate  and  non-federal  government  models  of  capital  budgeting 
do  not  adapt  well  to  the  federal  government.  While  adoption  of  capital  budgeting  reforms 
would  potentially  stabilize  the  funding  outlook  for  select  acquisition  programs,  such  actions 
inject  rigidity  into  a  process  that  demands  flexibility  in  so  many  other  areas.  Fiscal  law 
already  imposes  rigidity;  the  creation  of  special  accounts  for  acquisition  programs  simply 
adds  to  that  problem.  Capital  budgeting  reforms  are  unlikely  to  be  adopted  if  they  limit  broad 
analysis  of  the  defense  program  and  trade-offs  within  it.  Nor  will  adoption  be  likely  if  reforms 
challenge  the  balance  of  power  between  the  executive  and  legislative  branches.  Existing 
processes  already  encourage  the  employment  of  some  capital  budgeting  analytical 
processes;  the  specific  remedies  proposed  are  unlikely  to  resolve  the  problems  of  cost 
management.  Indeed,  they  may  create  new  problems. 

Rational  Cost  Model  Reforms 

The  second  area  of  reform  concerns  the  cost  estimate  on  which  the  investment 
decision  and  budget  are  based.  As  the  GAO  recently  stated,  “If  we  expect  programs  to  be 
executed  within  budget,  programs  need  to  begin  with  realistic  budgets.  The  foundation  of  an 
executable  budget  is  a  realistic  cost  estimate  that  takes  into  account  the  true  risk  and 
uncertainty  in  a  program”  (GAO,  2007,  p.  17).  There  is  widespread  dissatisfaction  with  the 
accuracy  of  defense  cost  estimating — both  in  Congress  and  within  the  DoD.  When  a 
program  experiences  cost  growth  above  its  budgeted  amount,  the  allocation  of  funds  in 
current  or  future-year  budgets  must  be  adjusted  to  keep  the  program  on  track;  doing  so 
requires  either  top-line  relief  or  a  decrease  to  one  or  more  other  program  budgets.  Either 
way,  expectations  are  unmet.  Before  evaluating  some  of  the  proposed  remedies  to  this 
problem,  we  must  understand  the  nature  of  the  problem. 

Of  course,  one  must  acknowledge  that  the  cost-growth  problem  involves  three 
variables.  First  is  the  cost  estimate  (E),  which  is  often  cited  as  a  discrete  figure,  but  in  reality 
is  a  range  of  values  defined  by  a  probability  distribution.  Second  is  the  amount  budgeted  (B) 
for  the  program,5  which  is  a  discrete  value  selected  presumably  from  somewhere  along  the 
cost-estimate  probability  curve.  Third  is  the  final  cost  of  the  program  (C),  a  discrete  value. 
The  public  often  scrutinizes  the  difference  between  B  and  C,  as  these  are  the  figures  that 
exist  in  budget  or  appropriation  documents,  contract  audits,  SAR  reports,  and  the  like. 
Scholars  and  management  reformers  tend  to  also  focus  on  the  relationship  between  E  and 
B,  but  the  nuances  often  fail  to  capture  the  public’s  and  politicians’  interest. 

Shown  graphically  in  Figure  1 ,  a  notional  cost-estimate  S-curve  (so  called  because 
of  its  shape)  will  define  a  probability  distribution  based  on  the  confidence  of  data  populating 
the  estimating  model  and  the  accuracy  of  the  model  itself  (E).  There  is  little  probability  of  the 
government  meeting  the  cost  estimate  at  the  low-dollar-value  end  of  the  curve,  but  a  very 
high  probability  of  meeting  program  goals  at  the  high-dollar-value  end  of  the  curve.  During 
the  budgeting  process,  a  value  is  selected  along  the  curve  at  the  appropriate  amount  of 


5  To  be  even  more  precise,  one  could  break  down  this  variable  into  the  programmed  amount,  the 
budgeted  amount,  the  appropriated  amount,  and  the  amount  actually  provided  to  the  program  office. 
Those  distinctions,  though  often  quite  real,  are  not  necessary  to  make  here. 
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funds — the  precise  point  on  the  curve  is  the  subject  of  some  reform  proposals  and  will  be 
discussed  shortly.  For  illustration  purposes,  the  50%  probability  value  is  shown  (B). 
Eventually,  at  program  completion,  one  has  a  final  cost,  notionally  shown  on  the  chart  at 
value  C.  Cost  growth  is  generally  considered  the  difference  between  B  and  C.  Many 
definitions  of  cost  growth  select  B  at  acquisition  Milestone  B  and  C  at  program  completion 
(or  latest  estimate  if  still  ongoing),  and  years  may  pass  between  the  formulation  of  B  and  the 
eventuality  of  C. 

Cost 


Probability 


Figure  1.  Cost  Growth  Diagram 


It  is  important  to  first  note  that  cost-estimating  errors,  when  they  exist,  are 
overwhelmingly  low  compared  to  the  cost  estimate.  This  author  has  yet  to  read  a  study  or 
hear  a  defense  official  or  politician  complain  that  cost  estimates  are  too  high  and 
overestimating  errors  are  a  problem.  While  this  may  seem  facetious,  it  is  a  critically 
important  point.  If  cost-estimating  errors  were  a  function  of  poor-quality  data  or  technically 
inaccurate  models,  then  one  would  expect  normally  distributed  errors  with  a  mean  near 
zero.  What  we  see  is  more  of  a  lognormal  distribution  with  very  few  observations  of 
overestimation — as  in  Figure  2,  below,  from  a  RAND  study.  This  distribution  suggests  that 
there  are  biases  in  the  data  or  models.  Thankfully,  biases  can  be  found  and  eliminated.  That 
is  what  much  of  the  research  and  many  of  the  recommendations  in  this  area  seek  to  do. 
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Figure  2.  Cost-growth  Factors  (CGF)  of  Major  Acquisition  Programs 

(Arena,  Leonard,  Murry  &  Younossi,  2006,  p.  22) 

What  are  the  biases  that  affect  cost  estimates?  RAND  cites  the  following  causes  for 
cost  growth:  “overoptimism,  estimating  errors,  unrecognized  technical  issues,  requirements 
creep,  lack  of  incentives  to  control  cost,  and  schedule  extensions”  (Younossi  et  al.,  2007,  p. 
xxi).  While  unrecognized  technical  issues  may  indicate  a  poor  estimate,  unrecognizab/e 
errors  do  not.  Most  cost-estimating  models  make  allowances  for  the  likely  ones;  the 
unknowable  ones  should  be  distributed  randomly,  as  should  errors.  Schedule  extensions 
and  requirements  creep  may  help  explain  why  final  costs  are  higher  than  the  estimate; 
however,  they  are  not  estimating  problems  if  the  changes  to  schedules  and  requirements 
occur  subsequent  to  the  estimate,  and  the  estimate  correctly  considered  the  original 
schedule  and  requirement.  Further,  it  is  incorrect  to  view  cost  growth  as  a  problem  in  a 
situation  in  which  requirements  grow,  and  the  cost  of  those  requirements — had  they  been 
considered — would  have  been  consistent  with  the  estimate.  In  those  cases,  the  item 
purchased  is  not  the  same  as  the  item  estimated.6  And  in  many  cases,  schedule  extensions 
are  an  effect,  not  a  cause,  of  cost  growth.  These  are  matters  of  program  management 
discipline,  not  cost-estimating  accuracy.  A  lack  of  incentives  to  control  costs  is  also  a  matter 
of  program  management  or  oversight  and  is  not  a  matter  of  cost  estimating,  directly.  It  may 
be  an  indirect  factor,  as  we  shall  see  below.  That  leaves  overoptimism  as  a  source  of  non¬ 
normal  bias  in  the  estimate. 

Melese  and  his  co-authors  blame  two  factors:  “bad  incentives  (psychological  and 
political-economic  explanations)  and  bad  estimation  (methodological  explanations)”  (Melese 
et  al.,  2007,  p.  359).  Similarly,  Flyvbjerg  (2006)  cites  three  categories  of  factors.  One 
(technical  estimating  errors)  he  dismissed  because  of  the  non-normal  distribution  of  errors 


6  It  is  important  here  to  acknowledge  that,  politically,  this  distinction  may  not  matter.  The  public  at 
large  does  not  hold  that  information  and  only  sees  that  the  latest  fighter  (or  ship  or  other  program) 
has  grown  in  cost.  It  does  not  see  that  the  airplane  that  was  bought  is  better  than  the  one  estimated. 
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and  the  general  lack  of  improvement  over  time.  The  other  two,  optimism  bias  and  strategic 
misrepresentation,7  are  characterized  as  psychological  and  political  factors,  respectively. 
Thus,  all  three  studies  identify  similar  biases,  but  they  suggest  different  remedies. 

The  problem  is  that  two  types  of  bias  exist  in  cost  estimating:  optimism  and  strategic 
misrepresentation.  These  biases  affect  the  cost  estimate,  the  amount  budgeted  for  the 
program,  or  both.  There  are  other  forces  that  affect  the  final  cost  figure  and  manifest  during 
the  execution  of  the  program:  requirements  creep  (including  the  unforeseen  technical 
issues),  some  schedule  extensions,  and  incentive  structures.  Let  us  consider  three 
proposed  remedies  to  the  cost-estimating  problem  in  light  of  these  biases  and  factors. 

DAPA  Recommendation — budget  higher  on  the  curve.  To  its  credit,  the  DAP  A  report 
did  not  look  solely  at  cost-estimating  errors,  but  rather  at  the  totality  of  issues  affecting 
defense  acquisition.8  As  part  of  its  authors’  comprehensive  examination,  recommendations 
were  offered  in  several  areas,  including  budgeting  and  cost  control.  Among  those  budgeting 
and  cost  control  recommendations  was  the  proposal  to  “Adjust  program  estimates  to  reflect 
‘high  confidence’ — defined  as  a  program  with  an  80%  chance  of  completing  development  at 
or  below  estimated  cost”  (Kadish  et  al.,  2006,  p.  13).  Acknowledging  the  overoptimism  and 
strategic  misrepresentation  biases  in  the  system,  which  they  refer  to  as  “the  conspiracy  of 
hope”  (p.  102),  the  authors  assert  that  using  optimistic  estimates  (defined  as  the  50% 
confidence  level)  results  in  excessive  restructuring  of  budgets  and  programs.  Essentially, 
the  DAPA  report  suggests  an  appropriate  level  of  funding:  that  B  should  be  set  at  a  specific, 
higher,  point  on  the  E  curve. 

There  is  an  attractive  logic  to  the  DAPA  proposal.  If  B  were  set  higher  on  the  E 
curve,  one  can  be  reasonably  certain  that  the  degree  of  cost  growth  would  diminish. 
Presumably,  only  1/5  of  programs  would  risk  cost  growth,  and  4/5  should  cost  approximately 
what  was  budgeted.  This  would  reduce  the  need  for  budgetary  and  programmatic 
adjustments,  as  far  fewer  programs  would  experience  growth  problems.  This  proposal, 
however,  results  in  higher  overall  spending  for  the  same  programs. 

Federal  appropriation  law  and  the  norms  of  the  federal  government  are  such  that 
every  dollar  appropriated  is  expected  to  be  spent.  If  a  program  is  funded  at  the  50% 
probability  level,  there  is  an  even  chance  it  will  either  cost  what  was  budgeted  or  will  cost 
more.  In  other  words,  there  is  nearly  a  100%  chance  it  will  cost  at  least  the  50%  level.  By 
funding  that  same  program  at  the  80%  probability  level,  there  is  zero  chance  that  it  will  cost 
the  amount  of  the  50%  or  the  60%  or  the  75%  estimate.  There  is  a  nearly  100%  probability  it 
will  cost  at  least  the  80%  estimate.  While  funding  at  the  higher  level  reduces  the  likelihood  of 
cost  growth,  it  does  so  by  guaranteeing  the  higher  cost  in  the  first  place.  There  is  no  longer 
the  possibility  of  a  negative  effect  on  other  programs;  it  is  foreordained.  Rather  than  risk  the 


7  Jones  &  Euske  define  strategic  misrepresentation  as  “the  planned,  systematic  distortion  or 
misstatement  of  fact — lying — in  response  to  incentives  in  the  budget  process”  (1991,  p.  437).  This 
definition  is  consistent  with  Flyvbjerg’s  use  of  the  term. 

8  The  report  warns  that  one  should  implement  comprehensive  change  rather  than  incremental 
change.  “Past  practices  are  replete  with  examples  demonstrating  that  if  you  adjust  one  part  of  the 
system  with  corrective  measures,  challenging  issues  surface  in  other  parts  of  the  system.  When 
untested  corrective  action  is  taken,  over  time  it  can  result  in  unintended  consequences”  (Kadish  et  al., 
2006,  p.  82).  This  author  agrees  wholeheartedly;  in  fact,  the  purpose  of  this  report  is  to  draw  attention 
to  those  unintended  consequences. 
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possibility  of  disrupting  secondary  programs  or  altering  schedules  and  reducing  quantities  of 
the  primary  program,  those  changes  necessarily  occur  at  the  outset. 

Melese  et  al.  Recommendation — TCE.  In  a  study  that  looked  specifically  at  cost¬ 
estimating  accuracy,  Melese  and  his  colleagues  employ  transaction  cost  economics  (TCE) 
theory  to  the  problem  of  cost  growth.  While  acknowledging  the  psychological  and  political 
biases  that  create  bad  incentives,  they  mainly  focus  on  improving  bad  estimation 
methodologies.  The  authors  suggest  the  use  of  TCE  will  obtain  less  biased  estimates  which, 
in  turn,  will  reduce  the  mean  and  variance  of  cost  growth.  Noting  that  inaccurate  estimates 
may  result  from  omitting  variables,  the  estimator  is  encouraged  to  consider  costs  beyond 
production  to  “include  coordination  and  motivation  costs  such  as  search  and  information 
costs,  decision  and  contracting  costs,  and  monitoring  and  enforcement  costs”  (Melese  et  al., 
2007,  p.  359).  After  clearly  describing  factors  within  the  DoD  and  their  contractors  that  drive 
cost  growth,  the  authors  conclude  that  “cost  estimating  techniques  must  properly  anticipate 
extra  transaction  costs  [...]  that  can  quickly  overwhelm  initial  production  cost  estimates”  (p. 
365). 


There  are  two  concerns  with  this  approach.  First,  “TCE  predicts  contracts  and  other 
governance  structures  will  be  chosen  that  reduce  transaction  costs  and  improve  the  gains 
from  exchange  between  buyers  and  sellers”  (p.  367).  This  is  true  only  so  far  as  both  buyer 
and  seller  are  motivated  by  a  concern  for  economic  efficiency.  This  is  not  necessarily  true 
when  one  of  the  actors  is  a  government.  Rules  governing  competitive  bidding,  free  trade, 
the  use  of  small  businesses,  and  Buy-American  provisions  all  add  transaction  costs  and 
often  raise  production  costs.  The  public  value  in  such  rules  is  not  economic;  it  is  elsewhere 
in  the  complexion  of  values  that  define  the  public  sector:  values  such  as  fairness,  equity, 
accountability,  or  justice.  The  evidence  is  not  at  all  clear  that  the  DoD  is  motivated  to  design 
governance  structures,  for  instance,  that  reduce  transaction  costs.  The  excessively 
bureaucratic  structure  of  the  acquisition  review  and  approval  processes  is  hardly  designed 
to  be  efficient.  Those  processes  are  arguably  designed  to  limit  undesirable  effects  more 
than  they  are  designed  to  encourage  desirable  ones. 

Second,  since  acquisition  cost  growth  is  measured  predominantly  in  production  cost 
terms,  the  increases  that  are  routinely  experienced  are  in  production  costs;  they  are  not 
caused  by  the  omission  of  transaction  costs.  Cost  growth  is  most  commonly  defined  as 
increases  in  development  and  production  costs  as  reported  in  Selected  Acquisition  Reports. 
In  other  words,  those  costs  are  the  ones  funded  through  RDT&E  and  procurement 
appropriations.  The  majority  of  the  transaction  costs,  on  the  other  hand,  are  funded  in 
operations  appropriations.  Contracting,  contract  administration,  auditing,  data  collection, 
oversight,  etc.,  are  neither  included  in  the  baseline  nor  in  the  final  cost  estimate.9  The  salary 
of  the  program  manager  is  not  even  included.  The  omission  of  such  costs  does  not  account 
for  the  growth. 

The  value  of  a  TCE  approach  comes  from  the  knowledge  that  managing  those 
activities  may  reduce  cost  growth,  but  including  them  in  the  estimate  will  not.  Inclusion  of 


9  Volume  2A,  Chapter  1  of  the  DoD  Financial  Management  Regulations  says:  “The  cost  of  civilian 
personnel  compensation  and  other  direct  costs  (i.e.,  travel,  office  equipment  leasing,  maintenance, 
printing  and  reproduction)  incurred  in  support  of  procurement  and/or  production  programs  by 
departmental  headquarters  staff,  contracting  offices,  contract  audit  offices,  system  project  offices,  and 
acquisition  managers  are  expenses.”  As  expenses,  they  are  not  funded  in  the  procurement  accounts. 
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those  costs  in  the  estimate  would  have  the  additional  unintended  consequence  of  ascribing 
an  economic  value  to  those  activities  above  the  other  social  values  they  were  intended  to 
create.  Audits  are  done  not  to  reduce  cost  but  to  provide  accountability  and  assurance  to  the 
public.  Governance  structures  exist  not  only  to  hold  economic  costs  down,  but  also  to 
ensure  against  waste  and  fraud  and  to  ensure  compliance  with  the  law.  Many  of  those  laws 
are  motivated  by  social,  not  economic  values.  Because  of  that,  decision-makers  should 
focus  not  on  minimizing  their  drain  on  the  acquisition  of  weapon  systems,  but  rather  on 
reducing  the  cost  of  attaining  that  social  goal  irrespective  of  the  weapons  system. 

Undoubtedly,  a  better  understanding  of  the  role  transaction  costs  play  in  the 
motivation  of  sellers  can  assist  in  the  negotiation  and  administration  of  contracts;  that  may, 
in  turn,  help  lower  the  cost  of  production.  But  because  many  of  those  transaction  costs 
serve  non-economic  goals,  they  should  remain  outside  the  cost  estimate  and  the  final  cost 
tally. 

Flyvbjerg  Recommendation — reference  class  forecasting.  There  is  another  way  to 
address  the  psychological  and  political  biases  other  than  using  TCE.  Flyvbjerg  (2006) 
recommends  using  reference  class  forecasting — an  estimating  methodology  based  on 
Kahneman  and  Taversky’s  (1979)  work  in  decision-making  under  conditions  of  uncertainty. 
Reference  class  forecasting  addresses  optimism  and  strategic  misrepresentation  biases  by 
relying  on  the  actual  performance  of  a  reference  class  of  comparable  projects.  To  combat 
the  optimistic  bias  inherent  in  an  “inside  view”  of  the  project,  under  reference  class 
forecasting,  those  estimating  the  cost  of  a  project  assume  an  “outside  view”  by  considering 
the  experiences  of  comparable  projects.  In  his  study  of  public  works  projects,  Flyvbjerg  finds 
that  different  classes  of  projects  improve  their  cost-estimating  accuracy  by  apply  various 
“uplifts”  to  the  inside  view  estimates.  Fie  finds  that  to  achieve  the  80%  probability  level  (the 
same  probability  the  DAPA  report  recommends),  estimates  for  road,  bridge/tunnel,  and  rail 
projects  should  rise  32,  55  and  57%,  respectively. 

Reference  class  forecasting  suffers  from  the  same  effect  as  the  DAPA 
recommendation:  improvements  in  cost-estimating  accuracy  are  achieved  by  forfeiting  the 
possibility  of  lower  total  costs.  Reference  class  forecasting  is  not  a  new  idea  for  defense 
acquisition.  The  “outside  view”  was  the  motivation  for  the  creation  of  the  Cost  Analysis 
Improvement  Group  in  1972  and  the  requirement  for  independent  cost  estimates  (Melese  et 
al.,  2007).  The  Congressional  Budget  Office  employs  this  basic  concept  in  their  report  series 
Long-term  Implications  of  Current  Defense  Plans  (CBO,  2007).  The  knowledge  exists.  One 
could  even  assume  it  is  considered  by  decision-makers;  it  simply  is  not  used.  The  important 
question  is  why  not? 

All  three  studies  suggest  remedies  to  improve  the  accuracy  of  cost  estimates  and  to 
lower  the  likelihood  of  cost  growth  in  defense  acquisition.  They  each  acknowledge  that  cost 
growth  is  less  a  technical  estimating  problem  as  it  is  a  problem  of  psychological  and  political 
bias.  The  three  recommendations  would  all  bring  B  closer  to  C  on  Figure  1  and  would  be 
able  to  claim  gains  in  cost-estimating  accuracy.  This  paper  concedes  that  such  methods  are 
likely  to  reduce  cost  growth,  but  it  also  warns  that  they  do  little  to  address  the  ultimate 
problem  of  total  cost.  In  fact,  all  three  recommendations  would  aggravate  rather  than 
mitigate  the  ultimate  problem:  high  costs  crowding  out  other  spending  and  their  effect  on 
military  force  structure.  Why?  Because  all  three  methods  program  those  costs  with  certainty 
rather  than  risk  the  possibility  of  them  occurring  later. 
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One  should  notice  that  all  three  methods  suggest  that  the  DoD  more  fully  consider 
costs  known  to  exist.  I  submit  that  those  costs  are  considered  and  intentionally  dismissed  for 
one  of  two  reasons.  One  reason — acknowledged  by  Melese  et  al.  and  Flyvberg — is  that  a 
lower  estimate  may  be  politically  necessary  to  achieve  the  “camel’s  nose  in  the  tent”  effect. 
Wildavsky  (1979)  suggested  that  low  initial  budget  estimates,  while  inaccurate,  serve  a 
useful  purpose  by  getting  a  program  initiated.  Then,  once  it  is  initiated,  upward  adjustments 
are  easier  to  obtain  in  the  future  than  approval  of  the  program  at  the  higher  amount  in  the 
first  place.  Strategic  misrepresentation  (Flyvbjerg,  2006;  Jones  &  Euske,  1991)  is  a  common 
occurrence  and  one  not  undertaken  lightly.  Those  engaging  in  strategic  misrepresentation 
weigh  the  benefit  of  a  higher  probability  of  program  initiation  against  the  higher  probability  of 
a  cost  to  one’s  reputation  and  the  inevitable  downstream  budget  pressure.  Evidently,  in 
some  cases,  the  balance  tips  in  the  favor  of  under-representing  cost. 

Another  reason  for  underestimating  is  that  the  Defense  Department  may  be  wittingly 
or  unwittingly  engaging  in  target  costing  (Monden  &  Hamada,  1991;  Cooper  &  Slagmulder, 
1997).  Target  costing  is  the  practice  of  intentionally  setting  aggressive  financial  targets  as 
an  inducement  to  achieve  those  targets.  It  is  done  with  full  knowledge  that  there  is  a  risk  of 
cost  growth,  but  that  risk  is  accepted  in  exchange  for  the  possibility  of  hitting  the  lower 
actual  cost.  Costs  are  managed  through  value  engineering  programs.  The  Navy’s  recent 
goal  of  producing  a  Virginia  class  submarine  for  $2  billion  is  an  example.  Target  costing  is 
the  opposite  of  what  the  three  studies  recommend.  Target  costing  can  be  effective  if  the 
environment  is  such  that:  (a)  program  managers  are  not  punished  for  missing  cost  goals  if 
their  actual  performance  is  reasonable,  and  (b)  resources  are  slack  enough  to  cover  the 
inevitable  cost  growth.  Both  conditions  tend  to  exist  in  the  DoD. 

Whether  the  reason  for  perpetuating  low  estimates  is  an  example  of  the  camel’s 
nose  theory  or  target  costing,  such  low  estimates  are  of  value  to  the  DoD.  In  the  first  case,  a 
desired  but  politically  risky  program  is  begun  and,  once  begun,  is  likely  to  perpetuate.  In  the 
second  case,  pressures  are  applied  to  programs  to  hold  costs  down,  and  the  possibility  of 
actually  hitting  those  ambitious  targets  is  left  open.  The  two  reasons  may  co-exist:  through 
ambitiously  low  estimates,  sufficient  resources  are  apparently  made  available  to  initiate  a 
new  program.  In  some  cases,  this  is  evidently  preferable  to  raising  the  estimate  of  the  first 
program  to  reduce  the  likelihood  of  cost  growth  and,  consequently,  to  eliminating  the 
possibility  of  initiating  the  second. 

Paradoxically,  it  is  the  dissatisfaction  with  cost  growth  that  permits  practices  like 
target  costing  to  work,  and  it  is  the  dissatisfaction  with  cost  growth  that  limits  the  use  of 
strategic  misrepresentation.  Funding  at  a  low  level  of  probability  can  be  revisited  during  the 
annual  budget  cycle,  but  in  the  presence  of  constrained  resources  and  dissatisfaction  with 
cost  growth,  program  managers  are  pressured  to  hit  those  ambitious  targets  and  minimize 
growth.  These  existing  practices  likely  hold  total  costs  lower  than  would  be  the  case  if  the 
recommendations  were  adopted.  Why?  Those  recommendations  relieve  the  pressure  to 
contain  cost  growth  but  do  so  in  exchange  for  nothing  other  than  the  satisfaction  of  having 
met  the  estimate.  Defense  behavior  has  shown  that  two  programs  bursting  at  the  budget 
seams  is  preferable  to  one  program  managed  comfortably. 

Conclusion 

Dissatisfaction  with  cost  growth  has  generated  a  set  of  recommendations  designed 
to  eliminate  that  growth.  Unfortunately,  adopting  an  inappropriate  or  unnecessary  model 
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may  not  be  helpful;  rigidity  is  not  helpful  when  flexibility  is  needed,  and  foreordaining  higher 
costs  is  not  helpful.  The  dissatisfaction  itself  is  helpful. 


Recall  that  the  main  concern  with  cost  growth  is  that  it  affects  the  total  defense 
program.  Systems  are  delivered  later  or  in  smaller  quantities  than  expected.  Root  causes  of 
the  growth  are  psychological  and  political  biases  that  manifest  in  the  behavior  of 
programmers,  program  managers,  budgeters,  contractors  and  politicians.  These  biases  exist 
for  a  reason,  and  accounting  for  them  does  not  make  them  disappear.  Accounting  for  them 
simply  makes  explicit  what  many  actors  in  the  system  know  implicitly.  Failing  to  account  for 
them  serves  to  add  pressure  and  incentives  to  the  system,  which  may  result  in  holding  costs 
lower  than  they  would  be  if  the  biases  were  accounted  for. 

The  underlying  concern  should  be  spending  in  total — not  spending  relative  to  an 
estimate.  However,  defense  leaders  and  stakeholders  should  not  be  complacent  about 
growth  because  their  dissatisfaction  serves  a  useful  purpose.  Rather  than  eliminate  that 
dissatisfaction,  it  should  be  understood  for  the  role  it  plays.  Indeed,  the  present  state  may 
very  well  have  evolved — not  unlike  Darwin’s  finches — to  achieve  a  satisfactory  balance 
among  all  the  forces  at  play:  in  this  case,  economic,  psychological  and  political  ones. 
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11:15  p.m.-  Chair: 
12:45  a.m. 


Stan  Z.  Soloway,  President,  Professional  Service  Council 

Discussant: 

CAPT  Stephen  H.  Huber,  Commander,  Port  Hueneme  Division,  Naval 
Surface  Warfare  Center 

Papers: 

Commercial-Off-the-Self  (COTS);  Doing  It  Right 

Dr.  William  Lucyshyn,  Director  of  Research  and  Senior  Research 
Scholar,  Center  for  Public  Policy  and  Private  Enterprise,  University  of 
Maryland 

Commercial-Off-the-Self  (COTS):  Impact  to  Reliability,  Maintainability  and 
Availability  (RM&A)  from  an  In  Service  Engineering  Perspective 

Robert  Howard,  Systems  Engineering  and  T&E  Division  Supportability 
Manager,  Land  Attack  Department,  Naval  Surface  Warfare  Center,  Port 
Hueneme  Division 


Chair:  Stan  Z.  Soloway  is  president  of  the  Professional  Services  Council — the  principal  national 
trade  association  representing  the  government  professional  and  technical  services  industry.  PSC  is 
known  for  its  leadership  on  the  full  range  of  government  acquisition/procurement  and  outsourcing  and 
privatization  issues.  Soloway  assumed  the  presidency  in  January  2001. 

Soloway  is  an  expert  on  the  relationship  between  the  public  and  private  sectors,  and  is  routinely 
sought  out  by  the  media,  federal  agencies,  congress  and  others  to  provide  commentary  and 
perspective  on  the  full  range  of  procurement  and  outsourcing  issues.  He  also  writes  a  monthly 
column  in  Washington  Technology  magazine,  and  was  a  member  of  the  congressionally  mandated, 
national  panel  on  the  future  of  government  outsourcing  chaired  by  the  Comptroller  General  of  the  US. 

Prior  to  joining  PSC,  Soloway  served  as  the  Deputy  Under  Secretary  of  Defense  (Acquisition  Reform) 
and  concurrently  as  director  of  Secretary  of  Defense  William  Cohen’s  Defense  Reform  Initiative.  As 
Deputy  Under  Secretary,  he  was  the  department’s  senior  official  responsible  for  the  development  and 
implementation  of  far-reaching  reforms  to  the  DoD’s  acquisition  processes  and  policies,  and  for  the 
oversight  of  the  training,  education  and  career  development  of  the  200,000-member  defense 
acquisition  workforce.  As  director,  DRI,  Soloway  led  significant  department-wide  re-engineering  and 
reform  initiatives  in  areas  as  diverse  as  privatization  and  outsourcing,  electronic  commerce,  financial 
management  reform,  logistics  transformation,  and  the  quality  of  life  for  American  troops. 

In  recognition  of  his  leadership  at  the  DoD,  Soloway  was  awarded  both  the  Secretary  of  Defense 
Medal  for  Outstanding  Public  Service  and  the  Secretary  of  Defense  Medal  for  Distinguished  Public 
Service. 


ACQUISITION  RESEARCH:  CREATING  SYNERGY  FOR  INFORMED  CHANGE  -  267  - 


In  2007,  Soloway  was  confirmed  by  the  US  Senate  for  a  seat  on  the  bi-partisan  Board  of  Directors  of 
the  Corporation  for  National  and  Community  Service.  Mr.  Soloway  is  a  principal  of  the  Council  on 
Excellence  in  Government,  and  was  an  expert  panelist  for  studies  conducted  by  the  Center  for 
Strategic  and  International  Studies  on  the  future  of  defense  technology  and  acquisition  policy.  He  is 
also  a  member  of  the  Board  of  Advisors  of  the  National  Contract  Management  Association,  was  a 
2005  recipient  of  the  prestigious  Federal  100  Award,  and  speaks  frequently  to  industry  and 
government  organizations  on  government  technology,  acquisition,  human  capital,  and  strategic 
management  issues. 

Before  his  appointment  to  the  DoD,  Soloway  was  a  public  policy  and  public  affairs  consultant  for  more 
than  20  years,  and  a  highly  regarded  expert  in  (and  frequent  lecturer  on)  acquisition,  privatization, 
and  outsourcing  issues.  He  also  co-produced  the  critically  acclaimed  “Great  Confrontations  at  the 
Oxford  Union,”  a  series  of  prime-time  specials  that  aired  nationally  on  public  television.  He  earned  a 
degree  in  Political  Science  from  Denison  University,  where  he  was  elected  to  the  National  Men’s 
Journalism,  National  Men’s  Leadership,  and  National  Political  Science  honorary  societies. 

Discussant:  Captain  Stephen  H.  Huber  was  born  in  West  Chester,  PA.  He  graduated  from  the  US 
Naval  Academy  and  was  commissioned  in  1980,  having  earned  a  Bachelor  of  Science  Degree  in 
Oceanography/Physics.  After  graduation,  Huber  served  as  a  Naval  Academy  Seamanship  and 
Navigation  Instructor,  followed  by  Surface  Warfare  Officers'  School  in  Coronado,  CA.  At  sea,  he  has 
served  aboard  USS  STEIN  (FF  1065)  as  Gunnery  Officer,  ASW  Officer  and  Navigator/Administrative 
Assistant;  USS  REASONER  (FF  1063)  as  Weapons  Officer;  and  USS  GARY  (FFG  51)  as  Executive 
Officer.  He  also  served  on  the  Afloat  Staff  of  Destroyer  Squadron  5  as  Operations  Officer,  and  on  the 
staff  of  Commander,  THIRD  Fleet,  as  Flag  Secretary.  Huber  was  Commanding  Officer  in  USS  FIFE 
(DD  991)  from  September  1998  through  April  2000.  During  his  command  tour,  FIFE  deployed  to  the 
Eastern  Pacific  in  support  of  Counter-Narcotics  Operations,  was  the  first  ship  to  go  through  an 
availability  using  a  private  contractor  in  a  public  shipyard,  and  was  awarded  the  SECNAV  Energy 
Conservation  Award.  Ashore,  Huber  has  served  as  Aide  to  the  Commandant,  Naval  District 
Washington,  DC,  and  on  the  staff  of  Commander  in  Chief,  US  Atlantic  Fleet,  as  the  Assistant  Surface 
ASW  Officer  and  Special  Operations  Officer. 

During  tours  in  Washington,  DC,  he  has  served  as  the  Combat  Systems  Training  Officer  on  the  staff 
of  the  Chief  of  Naval  Operations  and  at  NAVSEA,  in  the  Surface  Ship  Technology  Directorate  (SEA 
53)  and  as  Deputy  Program  Manager  in  PMS  430 — the  BFTT  program  office.  His  last  assignment 
was  as  Deputy  Director,  Human  Systems  Integration  Directorate  (SEA  03B). 

Huber  assumed  command  of  Port  Hueneme  Division,  Naval  Surface  Warfare  Center  in  May  2004. 
Under  his  command,  Port  Hueneme  Division  has  been  awarded  the  2006-2007  Shingo  Silver 
Medallion  in  the  Public  Sector  Category;  the  California  Award  for  Performance  Excellence,  Bronze 
level  2005  and  Silver  level  2006;  the  Deming  Award  for  Training  Excellence  2005;  the  Department  of 
Veterans  Affairs  Government  Contracting  Office/Facility  Award  during  2005;  was  the  NAVSEA 
Nominee  for  the  CNO  Safety  Award  2005;  the  2006  CNO  Environmental  Award;  and  received  the 
Industry  Partnership  Award  from  the  California  Regional  Consortium  for  Engineering  Advances  in 
Technological  Education  (CREATE),  2006.  A  strong  supporter  of  the  Navy's  Lean  Six  Sigma 
initiatives,  Huber's  leadership  has  resulted  in  recorded  savings  in  excess  of  $26  million  during  his 
tour. 

Huber's  awards  include  the  Meritorious  Service  Medal  (with  two  gold  stars),  the  Navy  Commendation 
Medal  (with  four  gold  stars),  the  Navy  Achievement  Medal,  and  several  unit  awards.  He  holds  a 
Master  of  Arts  degree  in  International  Studies  from  Old  Dominion  University,  and  a  Master  of  Arts 
degree  in  National  Security  and  Strategic  Studies  from  the  Naval  War  College.  He  was  inducted  into 
the  International  History  Honors  Society,  Phi  Alpha  Theta,  in  1993.  He  was  designated  an  Acquisition 
Professional  in  January  of  1997  and  DAWIA  Level  III  certified  in  Program  Management  in  February  of 
2002.  He  was  a  2003  National  Security  Studies  Fellow  at  the  Maxwell  School,  Syracuse  University. 


ACQUISITION  RESEARCH:  CREATING  SYNERGY  FOR  INFORMED  CHANGE  -  268  - 


Commercial-Off-the-Self  (COTS):  Impact  to  Reliability, 
Maintainability  and  Availability  (RM&A)  from  an  In  Service 
Engineering  Perspective 


Presenter:  Robert  Howard,  Systems  Engineering  and  T&E  Division  Supportability  Manager, 
Land  Attack  Department,  Naval  Surface  Warfare  Center,  Port  Hueneme  Division 


Abstract 

With  the  proliferation  of  COTS  within  Military  systems  based  on  DoD  acquisition 
directives,  RM&A  processes  and  products  requirements  during  acquisition  need  to  evolve  to 
ensure  systems  are  meeting  lifecycle  expectation.  RM&A  disciplines  and  products  have 
limited  focus  based  on  traditional  MIL-STD  architecture,  which  was  based  primarily  on 
hardware  failures  represented  by  the  traditional  bathtub  curve.  This  paper  discusses  some 
of  key  observations  and  solutions  being  applied  from  the  In-service  Engineering  Agent  (PHD 
NSWC)  along  with  lessons  observed  to  mitigate  future  risks. 

Acquisition  development  strategies  (such  as  incremental  and  spiral)  are  utilizing 
more  COTS  components  for  hardware  and  open  architecture  computing  strategies  for 
software.  This  drives  a  nasality  to  reevaluate  traditional  RM&A  efforts  performed  during 
concept,  development,  integration  and  fielding.  It  also  forces  decision-makers  to  assess  fleet 
feedback  systems  and  their  capability  to  provide  meaningful  data  to  understand  and  perform 
root-cause  analysis  on  issues  impacting  operational  availability,  manpower,  and  operational 
cost.  Lifecycle  RM&A  observations  were  made  on  the  level  of  accuracy  and  confidence 
required  to  support  COTS  Refresh  Selection,  changes  in  Duty  Cycle  and  their  relative 
impact  to  failure-rate  calculation,  reduced  applicability  of  the  Bath  Tub  Curve  for  RM&A 
prediction,  and  Human  System  Interface  (HSI)  impacts  to  failure  rate.  Through 
understanding  of  these  observations,  this  paper  provides  insight  into  reducing  and  avoiding 
introduction  and  sustainment  risks  and  costs  associated  with  COTS  and  OA. 

The  in-service  engineer  must  be  involved  through  government  oversight  to  verify 
both  that  system  requirements  are  articulated  and  captured  to  the  appropriate  level  to  reach 
the  desired  end-state  and  that  RM&A  analysis  tools  are  adapted  based  on  technology 
insertion.  During  development,  production,  and  systems  integration,  the  in-service  engineer 
must  ensure  OEMs  capture  data  and  indicators  which  could  identify  shortfalls  in  planned  or 
completed  analysis,  as  these  could  impact  the  expected  reliability,  maintainability,  or 
supportability  of  the  system.  Finally,  as  systems  are  fielded,  assessment  of  quantitative 
feedback  mechanism  must  be  complimented  with  qualitative  feedback  if  decision-makers 
are  to  identify  if  reporting  shortfalls  exist  for  the  receipt  of  fleet  feedback  on  RM&A  issues. 
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Panel  14  -  Issues  in  Cost  Estimating  for  US  Naval 
Shipbuilding 


Thursday, 
May  15,  2008 


11:15  a.m.-  Chair: 
12:45  p.m. 


Christopher  S.  Deegan,  Division  Director,  Cost  Engineering  and  Industrial 
Analysis  Division,  Naval  Sea  Systems  Command 


Papers: 

Submarine  Cost  Analysis 

LT  Ben  Grant,  Graduate  Student,  Naval  Postgraduate  School 

Dynamic  Cost  Risk  Assessment  for  Controlling  the  Cost  of  Naval  Vessels 

Dr.  Edouard  Kujawski,  Associate  Professor,  Jeffery  E.  Kline,  Program  Director  of 
Center  of  Executive  Education  and  Dr.  Diana  Angelis,  Associate  Professor,  Naval 
Postgraduate  School 

Using  the  Steel  Vessel  Material-cost  Index  to  Mitigate  Shipbuilder  Risk 

Edward  G.  Keating,  Senior  Economist,  Robert  Murphy,  Researcher,  John  F. 
Schank,  Senior  Operations  Research  Analyst  and  John  Birkler,  Research  Leader, 
Acquisition,  Defense  Production,  and  Technology  Base  Issues,  RAND  Corporation 


Chair:  Christopher  S.  Deegan,  Division  Director,  Cost  Engineering  and  Industrial  Analysis 
Division,  Naval  Sea  Systems  Command 
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Dynamic  Cost  Risk  Assessment  for  Controlling  the  Cost  of 
Naval  Vessels 

Presenter:  Dr.  Edouard  Kujawski  is  an  associate  professor  in  the  Systems  Engineering  Department  at  the 
Naval  Postgraduate  School.  His  research  and  teaching  interests  include  the  design  and  analysis  of  high 
reliability/availability  systems,  risk  analysis,  and  decision  theory.  He  received  a  PhD  in  theoretical  physics 
from  MIT,  following  which  he  spent  several  years  in  research  and  teaching  physics.  He  has  held  lead 
positions  at  General  Electric,  Lockheed-Martin  and  the  Lawrence  Berkeley  National  Laboratory.  He  has 
contributed  to  the  design  of  particle  accelerators  and  detectors,  space  observatories,  commercial 
communication  systems,  the  Space  Station,  and  nuclear  power  plants.  He  was  a  participant  and  contributor 
to  the  Lockheed  Martin  LM21  Risk  Management  Best  Practices  and  the  original  INCOSE  Systems 
Engineering  Handbook.  He  is  a  member  of  the  San  Francisco  Bay  Area  Chapter  of  INCOSE  and  has  served 
on  the  board  of  directors. 

Edouard  Kujawski 
Systems  Engineering  Department 
Naval  Postgraduate  School 
Monterey,  California  93943 

Author:  Dr.  Diana  Angelis  is  an  Associate  Professor  in  the  Defense  Resources  Management 
Institute  at  the  Naval  Postgraduate  School  in  Monterey,  CA.  She  joined  the  faculty  in  1996.  She 
studied  accounting  at  the  University  of  Florida  and  received  a  BS  in  Business  Administration  in  1977 
and  a  BS  in  Electrical  Engineering  in  1985.  She  received  her  PhD  in  Industrial  and  Systems 
Engineering  from  the  University  of  Florida  in  1996.  Her  research  interests  include  the  application  of 
activity-based  costing  in  government  organizations,  cost  estimating,  the  valuation  of  R&D  through 
options  theory,  and  business  reforms  in  defense  management.  She  was  commissioned  an  officer  in 
the  United  States  Air  Force  in  1984  and  served  as  a  program  engineer  until  1989.  She  joined  the 
USAF  Reserves  in  1990  and  has  worked  in  both  acquisition  and  test  &  valuation  with  the  Air  Force 
Materiel  Command.  Dr.  Angelis  is  a  Certified  Public  Accountant  and  a  Lieutenant  Colonel  in  the  US 
Air  Force  Reserve  currently  assigned  to  the  Air  Force  Flight  Test  Center  at  Edwards  AFB,  CA. 

Diana  Angelis 

Defense  Research  Management  Institute 
Naval  Postgraduate  School 
Monterey,  California  93943 


Abstract 

Naval  vessels,  like  most  large-capital  projects,  have  a  long  history  of  cost  growth.  To 
get  a  handle  on  this  problem,  NAVSEA’s  Cost  Engineering  &  Industrial  Division,  NAVSEA 
05C,  has  introduced  Probabilistic  Cost  Risk  Analysis  (PCRA)  into  the  Department  of 
Defense  (DoD)  Planning,  Programmatic,  Budgeting,  and  Execution  System  (PPBES).  The 
quantification  of  cost  in  terms  of  cumulative  probability  distribution  functions  (CDF)  or  “S- 
curves”  provides  a  macroscopic  view  of  project  risk.  Risk  curves  alone  do  not  provide 
adequate  visibility  into  the  individual  project  risk  drivers;  therefore,  they  are  insufficient  for 
planning  and  managing  risk  reduction  activities  (RRA).  Complex  projects  typically  involve  a 
set  of  high-consequence,  project-specific  risks  that  require  detailed  analysis  and  for  which 
risk  response  actions  need  to  be  developed  and  implemented.  The  analysis  of  specific  risks 
and  RRAs  requires  a  microscopic  view.  We  present  a  practical  and  mathematically  sound 
approach  using  scenarios  and  Monte  Carlo  simulation  within  the  framework  of  decision  trees 
and  risk  curves.  The  approach  is  detailed  using  a  realistic  but  simplified  case  of  a  project 
with  three  technical  risks. 
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Introduction 


Cost  growth  has  been  a  major  problem  for  the  US  Navy.  Over  the  past  four 
decades,  the  growth  of  US  Navy  ship  costs  has  exceeded  the  rate  of  inflation.  In  the  past 
50  years,  annual  cost  escalation  rates  for  amphibious  ships,  surface  combatants,  attack 
submarines,  and  nuclear  aircraft  carriers  have  ranged  from  7  to  11%  (Arena,  Blickstein, 
Younossi  &  Grammich,  2006).  Along  with  real  cost  growth,  the  DoD  has  had  significant 
problems  with  cost  estimates.  By  and  large,  the  DoD  and  the  military  departments  have 
underestimated  the  cost  of  buying  new  weapon  systems.  A  recent  study  by  RAND  (Arena, 
Leonard,  Murray  &  Younossi,  2006)  indicates  that  there  is  a  systematic  bias  toward 
underestimating  weapon  system  costs  and  substantial  uncertainty  in  estimating  the  final 
cost  of  a  weapons  system. 

The  DoD  recognizes  that  uncertainty  is  an  important  part  of  cost  estimating.  During 
a  2007  seminar  with  a  naval  aviation  program  official,  the  Assistant  Secretary  of  the  Navy  for 
Research,  Development  and  Acquisition,  Dr.  Etter  (Burgess,  2007),  stated: 

Program  managers  not  only  need  to  know  a  realistic  cost  estimate  for  their  program, 
they  need  to  know  the  percent  probability  of  achieving  that  target.  For  example,  a 
ship  with  a  40%  chance  of  coming  in  on  budget  has  a  60%  chance  of  being  over 
budget.  Such  a  situation  should  prompt  the  project  manager  to  seek  help  from  the 
acquisition  community,  (p.  42) 

There  is  an  ongoing  major  shift  in  R&D  and  complex  engineering  projects  from 
deterministic  to  probabilistic  approaches.  Probabilistic  Cost  Analysis  (PCA)  provides  the 
proper  framework  for  handling  the  many  different  elements  of  cost  uncertainty,  including 
project-specific,  high-consequence  risks.  These  risk  drivers  must  be  identified,  assessed, 
mitigated,  and  controlled  through  formal  risk  management — which  is  an  essential  and  critical 
discipline  implemented  in  today’s  DoD  projects.  The  Risk  Management  Guide  for  DoD 
Acquisition  (2006)  reads: 

Risk  management  is  a  continuous  process  that  is  accomplished  throughout  the  life 
cycle  of  a  system.  It  is  an  organized  methodology  for  continuously  identifying  and 
measuring  the  unknowns;  developing  mitigation  options;  selecting,  planning,  and 
implementing  appropriate  risk  mitigations;  and  tracking  the  implementation  to  ensure 
successful  risk  reduction.  Effective  risk  management  depends  on  risk  management 
planning;  early  identification  and  analyses  of  risks;  early  implementation  of  corrective 
actions;  continuous  monitoring  and  reassessment;  and  communication, 
documentation,  and  coordination,  (p.  3) 

The  DoD  considers  the  definition,  implementation  and  documentation  of  risk 
management  essential  to  acquisition  success.  The  DoD  risk  management  process  outlined 
in  the  Risk  Management  Guide  consists  of  the  following  five  activities  performed  on  a 
continuous  basis:  Risk  Identification,  Risk  Analysis,  Risk  Mitigation  Planning,  Risk  Mitigation 
Implementation,  and  Risk  Tracking. 

This  process  is  consistent  with  the  AACE  definition,  which  includes  identifying  and 
analyzing  risk  factors  or  drivers,  mitigating  the  risk  drivers  where  appropriate,  estimating 
their  impact  on  plans  and  monitoring  and  controlling  risk  during  execution  (Hollman,  2006). 
To  be  effective,  PCA  must  interface  with  each  of  the  risk  management  activities. 
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An  emphasis  on  risk  management  supports  efforts  to  reduce  lifecycle  costs  of 
system  acquisitions.  An  often-neglected  concept  in  project  risk  management  is  the 
consideration  of  the  entire  project  lifecycle.  Analysis  of  risk  over  the  lifecycle  of  a  system 
can  yield  substantial  benefits.  Conversely,  ignoring  important  stages  of  the  lifecycle  can 
lead  to  substantial  problems  in  terms  of  risk  for  product  development  at  the  beginning  of  the 
lifecycle  and  for  product  upgrade  or  replacement  at  the  end  (Pennock  &  Haimes,  2001 ). 

Many  sources  of  cost  uncertainty  in  naval  vessel  construction — such  as 
economic/business  factors  (rates-wages,  overhead,  G&A,  vendor/supplier  stability,  inflation 
indices,  multi-year  assumptions,  etc.),  learning/rate/curve  assumptions,  and  cost- reduction 
initiatives — are  well  understood  within  the  framework  of  a  macroscopic  perspective;  these 
are  effectively  modeled  with  classical  Probability  Distribution  Functions  (PDF)  such  as  the 
triangular,  Beta,  lognormal,  and  Weibull  distributions.  However,  these  factors  constitute  only 
a  fraction  of  today’s  typical  project  risk  drivers  and,  therefore,  cost  uncertainty. 

The  construction  of  naval  vessels,  like  most  complex  engineering  projects,  is  also 
susceptible  to  project-specific  risk  drivers,  such  as:  low  Technology  Readiness  Level  (TRL); 
high  design,  manufacturing,  and  complexity;  significant  requirement  changes;  sizeable 
quantity  changes;  large  funding  uncertainty;  severe  acts  of  nature;  and  serious  accidents. 

It  is  tempting  to  assume  or  claim  that  the  PDFs  typically  elicited  for  cost  elements 
also  quantify  the  project-specific,  high-consequence  risks.  Sometimes  cost  analysts  will  go 
through  the  effort  of  identifying  and  discussing  risk  drivers,  but  when  it  comes  to  quantifying 
the  risks  and  estimating  contingency,  they  simply  apply  high/low  ranges  to  WBS  elements 
without  thinking  about  how  a  particular  risk  driver  affects  one  or  more  cost  elements.  We 
think  it  is  invalid  and  counterproductive  to  do  this  because  it  leads  to  the  loss  of  valuable 
information  and  visibility  into  these  risks.  Also,  this  approach  tends  to  focus  on  cost 
reduction  rather  than  risk  mitigation.  Hollmann  (2007)  notes  that  in  best  practice,  risk 
analysis  should  begin  with  the  identification  of  risk  drivers  and  events.  The  cost  impacts  of 
the  risk  drivers  and  events  are  then  considered  specifically  for  each  event. 

The  analysis  of  specific  risks  and  Risk  Response  Actions  (RRA)  requires  a 
microscopic  view  and  is  best  carried  out  with  tools  such  Decision  Trees  (DT),  influence 
diagrams,  or  other  discrete  representations.  This  microscopic  perspective  offers  many 
benefits.  It  is  a  powerful  risk  analysis  method  to  explicitly  model  high-consequence  risks 
and  RRAs,  and  thereby  provides  a  tool  for  making  better  decisions.  It  also  assists  subject- 
matter  experts  (SMEs)  to  think  about  credible,  high-consequence  events  and  better  deal 
with  overconfidence  or  optimism  biases.  However,  the  microscopic  view  is  too  cumbersome 
to  individually  analyze  every  risk  and  source  of  cost  uncertainty.  It  complements  and  needs 
to  be  integrated  within  the  PCA. 

In  this  paper,  we  propose  to  develop  a  microscopic/macroscopic  PCA  as  an  integral 
entity  of  the  DoD  risk  management  process,  as  follows: 

1 .  The  cost  and/or  risk  analyst  (simply  referred  to  as  analyst  below)  and  the  SMEs 
jointly  identify  the  individual  risks  using  the  standard  DoD  risk-identification  process. 

2.  The  analyst  and  the  SMEs  jointly  screen  the  identified  risks  for  further  analysis  and 
risk  mitigation. 

3.  The  analyst  and  SMEs  jointly  identify  realistic  RRAs  for  the  screened  risks. 
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4.  The  analyst  models  each  risk  and  its  RRAs  using  a  DT. 

5.  The  analyst  works  with  the  SMEs  to  quantify  the  value  of  the  decisions  and 
outcomes  for  each  DT  using  discrete  and  continuous  distributions.  We  favor  the 
Direct  Fractile  Assessment  (DFA)  method  for  data  elicitation  and  fitting  the 
associated  cost  elements  with  a  three-parameter  Weibull  distribution. 

6.  The  analyst  quantifies  the  DTs  using  Monte  Carlo  simulation.  Risks  and  RRAs  are 
then  modeled  in  terms  of  risk  curves.  We,  thereby,  avoid  relying  on  the  minimum 
expected  risk  value,  which  is  a  serious  shortcoming  of  standard  decision  analysis. 

7.  The  analysis  is  readily  performed  using  commercial  Excel  add-ins  (Crystal  Ball, 
@Risk...)  or  more  specialized  tools  (DecisionPro,  Analytica...). 

The  goal  of  this  paper  is  to  present  a  realistic  and  practical  method  for  explicitly 
analyzing  and  controlling  the  cost  impact  of  project  risks  and  realistic  RRAs.  Projects  can 
then  dynamically  determine  the  optimal  temporal  set  of  decision  gates  for  a  given  probability 
of  success — thereby  reducing  cost  while  increasing  the  probability  of  project  success.  We 
illustrate  the  method  using  a  realistic  but  simplified  case  of  a  project  with  three  technical 
risks.  We  close  with  some  concluding  remarks  and  recommendations  for  further 
development. 

The  Quantification  of  Multiple  Project  Risks 

Consider  a  project  with  n  credible,  high-consequence  risks  {Ri}.  Each  risk,  Rt,  is 
characterized  by  a  probability  of  occurrence  Pi  and  a  spectrum  of  possible  outcomes  with  a 
PDF  Lj(x),  where  x  is  a  random  variable  that  represents  the  magnitude  of  the  associated 
cost  or  loss.  One  may  then  think  of  this  set  of  risks  as  a  risk  portfolio  or  repository  (Kujawski 
&  Miller,  2007)  with  a  generalized  discrete  PDF  Rs(x)  given  by: 


Pi  >0^|  (1) 

The  total  project  cost  is  a  random  variable  that  consists  of  the  sum  of  the  m  base 
cost  elements  {BCJ  and  the  explicitly  identified  risk  costs  {RCJ.  Depending  on  the  state  of 
knowledge  of  the  data,  the  base  cost  elements  BC,  may  be  modeled  as  either  point 
estimates  or  continuous  PDFs.  The  total  project  cost  TC  is  then  the  probabilistic  sum  of  the 
m  base  cost  elements  and  n  risk-driver  costs: 


R-sO)  =  j(Pi>  Li(x)Mp2>  L2(x))>--->(Pn>  Ln(x)),^l-£ 


TC(x)  =  J  BC(x)  +  J  p,L(x)  (2) 

/=i  i=i 

Equations  (1)  and  (2)  provide  visibility  into  the  link  between  the  credible,  high- 
consequence  risks  {RJ  and  the  total  project  cost-risk  curve.  Monte  Carlo  simulation  tools 
such  as  Crystal  Ball  and  @Risk  can  also  provide  tornado  charts  that  conveniently  quantify 
the  importance  of  the  various  risk  drivers  and  their  link  to  the  overall  cost  risk.  Projects  can 
use  this  information  to  rationally  identity  risks.  This  is  in  sharp  contrast  with:  (1)  the  use  of 
point  estimates  that  are  at  best  ambiguous  because  overly  confident  staff  provide  low  cost 
estimates,  while  others  may  inflate  their  cost  estimates  to  make  it  easier  to  achieve  success, 
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(2)  decision-making  based  on  qualitative  assessments,  and  (3)  the  consideration  of  only  S- 
curves,  which  only  provide  a  macroscopic  and  somewhat  “black  box”  view  of  project  risk  and 
cost  uncertainty. 

Modeling  and  Analyzing  Risk  Response  Actions 

We  model  and  analyze  each  screened  risk  and  the  proposed  RRAs  using  a 
generalized  DT — where  PDFs  rather  than  discrete  branches  are  associated  with  the  chance 
nodes,  and  the  outcomes  are  analyzed  using  Monte  Carlo  simulation  (Kujawski,  2002). 

This  provides  a  powerful  technique  for  dealing  with  the  complex  situations  typical  of  today’s 
DoD  projects.  It  avoids  bushy  trees  and  generates  risk  curves,  thereby  removing  the 
reliance  of  decision-making  based  on  expected  value. 

To  illustrate  the  approach,  consider  the  risk  depicted  in  Figure  1 .  To  be  concrete, 
Risk  #1  is  associated  with  fabricating  a  complex  module.  The  two  risk  response  actions  are: 
(i)  Directly  fabricate  the  module,  or  (ii)  Build  a  prototype  and  then  fabricate  the  module.  The 
generalized  DT  follows  the  standard  DT  representation.  Decision  nodes  and  chance  nodes 
are  depicted  as  squares  and  circles,  respectively.  The  branches  that  originate  with  decision 
nodes  represent  the  available  RRAs.  The  branches  that  originate  with  chance  nodes 
represent  the  possible  probabilistic  outcomes.  A  descriptive  label,  a  probability,  and  a  cost 
distribution  are  associated  with  each  branch.  These  probability  and  cost  values  are 
conditional  on  the  RRA  and  may  also  be  conditional  on  the  outcome  of  other  risks  in  case  of 
interdependencies.  We  model  the  cost  values  using  a  three-parameter  Weibull  distribution 
fitted  to  the  10th,  50th,  and  90th  percentiles  determined  in  accordance  with  the  Direct  Fractile 
Assessment  (DFA)  method. 

In  this  example,  we  assume  that  the  baseline  cost  is  $1,1 00K.  Risk  is  then  given  by 
the  Value  At  Risk  (VAR)  relative  to  this  value.  The  VAR  corresponds  to  the  events  whereby 
production  of  the  module  exceeds  $1,1 00K.  The  ordering  of  the  decision  nodes 
corresponds  to  different  temporal  deterministic  events  in  the  development  and  fabrication 
cycle  of  the  module. 
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NOTE:  In  this  hypothetical  case,  the  values  may  be  thought  of  as  $K. 
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Figure  1.  Generalized  Decision  Tree  for  Risk  #1  and  Two  Initial  Candidate  Risk 

Response  Actions 

We  evaluated  each  RRA  in  Figure  1  using  the  Excel  Monte  Carlo  simulation  add-in, 
Crystal  Ball.  The  selection  of  a  RRA  is  a  deterministic  event,  and  only  the  associated 
outcomes  can  be  realized.  It  would,  therefore,  be  inappropriate  to  weigh  or  combine  the 
outcomes  of  the  two  RRAs  since  they  are  mutually  exclusive.  The  PDFs  and  risk  profiles  for 
each  individual  RRA  at  the  start  of  the  project  are  depicted  in  Figures  2a  and  2b, 
respectively.  The  PDFs  are  multimodal  and  cannot  be  represented  using  any  of  the  well- 
known  probability  distribution  functions.  The  peak  for  the  “prototype”  RRA  corresponds  to 
the  outcome  in  which  the  fabrication  of  the  module  fails.  The  PDF  for  the  “direct  fabrication” 
RRA  has  two  modes  corresponding  to  the  sequence  of  events  in  which  the  first  fabrication 
and  the  subsequent  fabrication  following  redesign  both  fail. 

The  Complementary  Cumulative  Distribution  Functions  (CCDF)  or  risk  curves  are 
shown  in  Figure  2b,  in  which  the  exceedance  probability  is  the  probability  of  exceeding  a 
given  consequence  or  (1  -  the  probability  of  success).  For  example,  looking  at  the 
VAR(Fab_A2)  curve  in  Figure  2b,  one  reads  that  there  is  approximately  a  30%  probability 
that  the  cost  will  exceed  $1 ,500K.  Equivalently,  one  can  state  that  there  is  a  70%  probability 
that  the  cost  will  be  less  than  $1 ,500K.  The  risk  curve  and  the  cumulative  distribution 
function  carry  identical  information  content.  Since  we  are  focusing  on  specific  risks  and 
VAR,  we  favor  the  risk  curve  or  CCDF  because,  in  our  opinion,  it  provides  a  better  view  of 
the  residual  risk  and  management  reserve  than  the  S-curve  (or  CDF)  that  typical  represents 
the  total  cost  (including  the  baseline  and  risk  cost  elements). 

For  any  given  value  on  the  x-axis,  the  risk  curve  that  corresponds  to  the  lowest 
exceedance  probability  represents  the  lower  risk.  Figure  2b  illustrates  that  the  prototype  risk 
curve  is  significantly  lower  than  the  fabrication  risk  curve  and,  thus,  has  less  risk.  In  this 
hypothetical  but  realistic  situation,  the  investment  of  $100K  for  building  a  prototype  provides 
a  significant  return  on  the  investment  as  measured  by  the  significant  risk  reduction.  To  be 
more  precise,  the  prototype  RRA  presents  a  lower  cost  of  risk  mitigation  for  all  values 
greater  than  $200K.  For  the  manager  trying  to  decide  if  it  is  worthwhile  to  invest  in  the 
prototype  option,  the  answer  is  to  invest  as  long  as  the  anticipated  benefits  from  the 
prototype  (whether  it  be  cost  savings,  time  savings,  information,  etc.)  exceed  $200K. 


Risk  #1 ,  RRA:  Prototype  module 


100  500  900  1,300 

Value  at  risk,  $K 


NOTE:  Given  the  different  scales,  the  two  PDFs  are  shown  separately  for  greater  visibility. 

Figure  2a.  Probability  Distributions  Corresponding  to  the  Two  RRA  Options 

for  Risk  #1 


Risk  #1 ,  RRA:  Fabricate  module 
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Risk  #1 ,  RRA  risk  curves 


Figure  2b.  Risk  Curves  for  the  Two  RRA  Options  for  Risk  #1 

The  Dynamic  Character  of  Risk  Response  Actions 

As  a  project  progresses,  its  risk  picture  is  dynamic.  The  sources  and  consequences 
of  risks  continue  to  evolve  and  change  over  time.  As  more  information  is  obtained  about  a 
particular  risk,  the  RRA  options  might  change;  thus,  it  is  necessary  to  constantly  monitor 
risk.  In  general,  at  any  point  in  time  there  will  be  a  mix  of  acceptable  and  unacceptable 
results.  The  performance  of  the  RRAs  should  be  monitored  and  controlled  to  ensure  they 
are  adequately  mitigating  risk.  Concurrently,  management  reserves  should  be  reviewed  on 
a  periodic  basis  and  dynamically  allocated  where  needed  to  ensure  project  success.  The 
Lockheed  Management  Student  Guide  (1998,  p.  33)  states,  “Risk  management  efforts  that 
fail  do  so  because  the  risk  control  actions  did  not  keep  up  with  a  changing  program 
situation.” 


As  discussed  in  the  previous  section,  we  use  risk  DTs  to  model  the  evolution 
of  the  potential  RRAs.  For  example,  Figure  3  depicts  the  Risk  #1  risk  curves  at  the  start  of 
implementation  of  the  “Prototype”  RRA  and  after  the  successful  demonstration  of  the 
prototype.  The  latter  risk  curve  moves  to  the  left  of  the  original  risk  curve  and  is  narrower, 
which  reflects  a  reduced  risk.  These  two  risk  curves  represent  the  value  of  the  unmitigated 
risk  exposure  at  two  different  points  in  time  and,  thereby,  provide  a  metric  for  the  risk 
exposure  characteristics.  This  information  is  essential  if  analysts  are  to  track  the  value  of 
the  residual  exposure  versus  the  value  or  cost  of  the  expended  RRAs  and  modify  the  RRAs 
as  needed  to  ensure  mission  success.  Note  that  if  the  risk  curve  moves  to  the  right  of  the 
original  risk  curve,  it  means  that  risk  exposure  is  increasing,  and  RRAs  need  to  be  re¬ 
evaluated. 
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Risk  #1 ,  Prototype  RRA 
Risk  exposure  characteristics 
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Figure  3.  Risk  Exposure  Characteristics  for  Risk  #1  with  Development  of  a  Prototype 
for  Risk  Mitigation  at  the  Start  and  after  Successful  Demonstration 

Application  to  a  Project  with  Multiple  Risks 

Now  consider  the  hypothetical  project  with  the  following  three  independent  risks: 

Risk  #1  depicted  in  Figure  1 ;  Risks  #2  and  #3  depicted  in  Figure  4a  and  4b,  respectively.  It 
is  both  rich  and  simple  enough  to  illustrate:  (1)  several  diverse  RRAs  and  their  analysis,  (2) 
the  dynamic  nature  of  the  risk  picture,  and  (3)  the  monitoring  of  individual  risks  and 
allocation  of  management  reserves.  The  approach  readily  extends  to  dependent  risks  using 
different  probability  and  outcome  values  that  reflect  causality  effects  among  the  risks. 

Figure  4a  may  be  thought  of  as  the  prime  contractor  subcontracting  the  engineering 
and  fabrication  of  a  complex  module.  The  prime  is  considering  the  following  two  options:  (1) 
subcontract  to  a  single  contractor  A,  denoted  by  the  branch  PDR_A  associated  with  the 
initial  node;  (2)  carrying  two  subcontractors  and  selecting  the  best  one  for  fabrication  at  the 
Preliminary  Design  Review  (PDR).  The  labeling  is  somewhat  cumbersome  because  each 
branch  needs  to  be  uniquely  identified.  The  PDR_A  sequence  represents  the  decision  to 
proceed  with  a  single  contractor.  The  PDR_AB  sequence  represents  the  decision  to 
proceed  with  two  contractors  and,  at  PDR,  to  select  the  best  one  for  manufacturing.  By 
selecting  two  different  contractors  with  different  offerings,  the  prime  significantly  reduces  the 
probability  of  PDR  failure.  RW  represents  the  cost  associated  with  rework;  it  is  modeled 
with  a  three-parameter  Weibull  distribution  specified  in  terms  of  the  90th,  50th,  and  10th 
percentiles  provided  by  SMEs  or  historical  data. 

Figure  4b  may  be  thought  of  a  prime  contractor  who  considers  two  different 
Verification  and  Validation  (V&V)  strategies  as  a  means  for  risk  reduction.  The  branch 
VVS_1_(Start  or  CDR)  represents  the  use  of  the  standard  approach  with  planned 
expenditures  of  $300K.  The  branch  VVS_2_  (Start  or  CDR)  represents  the  use  of  a  more 
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thorough  V&V  strategy  with  greater  use  of  simulation  and  planned  expenditure  of  $1 ,000K. 
The  branch  RW_1_(PDR  or  CDR)  represents  the  rework  following  the  PDR  and  CDR, 
respectively.  The  rework  is  assumed  to  be  inversely  related  to  the  V&V  effort,  and  it  is 
modeled  with  a  three-parameter  Weibull  distribution  specified  in  terms  of  the  90th,  50th,  and 
10th  percentiles  provided  by  SMEs  or  historical  data. 


t/ift  ftW_A2 


Figure  4a.  Generalized  Decision  Tree  for  Risk  #2  and  Two  Initial  Candidate  Risk 

Response  Actions 
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NOTE:  The  start  (or  PDR)  and  CDR  periods  are  shown  separately  to  simplify  the 
representation  of  the  sequence  of  events. 
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Figure  4b.  Generalized  Decision  Tree  for  Risk  #3  and  Two  Initial  Candidate  Risk 

Response  Actions 

Given  the  above  illustrative  project — with  three  risks  each  with  two  potential  RRAs — 
there  are  eight  possible  initial  Total  Project  RRAs  (TPRRA).  As  previously  discussed,  the 
risk  picture  is  dynamic  and  gets  quite  complex  as  through  time.  Consider  Risk  #1  with 
development  of  a  prototype  as  a  risk  reduction  option.  The  prototype  may  fail  or  succeed, 
and  the  fabrication  of  the  final  module  may  fail  or  succeed.  The  full  representation  of  the  set 
of  all  possible  outcomes  for  even  this  project  is  overwhelming  and  beyond  the  scope  of  a 
symposium  paper.  We,  therefore,  limit  ourselves  to  reporting  an  interesting  subset  of  the 
complete  analysis  as  follows: 

1 .  We  consider  only  two  of  the  eight  TPRRAS. 

a.  Strategy  1 .  Use  of  the  lowest  cost-mitigation  option  for  each  risk,  which  is 
equivalent  to  proceeding  as  normal — i.e.,  no  specific  RRA  for  any  of  the  three 
risks.  This  is  the  approach  that  a  risk-seeker  project  manager  would  favor. 

b.  Strategy  2.  Use  the  most  effective  RRA  for  each  risk.  This  corresponds  to:  (1 ) 
developing  a  prototype  for  Risk  #1 ,  (2)  proceeding  with  two  contractors  for  Risk  #2, 
and  (3)  implementing  a  more  thorough  V&V  effort  for  Risk  #3.  This  is  the  approach 
that  a  risk-averse  project  manager  would  favor. 

2.  For  each  strategy,  we  assume  the  best  possible  outcomes  for  the  probabilistic  nodes 
through  time  T1:  the  Risk  #1  prototype  and  the  Risk  #2  review  PDR_AB  succeed. 
Risk  #3  has  no  gates;  the  risk  reduction  is  directly  accounted  in  the  magnitude  of  the 
rework.  Figure  5  compares  the  initial  and  residual  risks  under  the  two  strategies. 

3.  For  each  strategy,  we  assume  the  worst  outcomes  for  the  probabilistic  nodes 
through  time  T1:  the  Risk  #1  prototype  and  the  Risk  #2  review  PDR_AB  fail.  Risk  #3 
has  no  gates;  the  risk  reduction  is  directly  accounted  in  the  magnitude  of  the  rework. 
Figure  6  compares  the  initial  and  residual  risks  under  the  two  strategies. 

4.  For  convenience,  we  also  report  the  50th,  80th,  and  mean  values  for  the  aggregated 
risks  and  individuals  risks  for  strategies  1  and  2  in  Tables  1  and  2,  respectively. 

Useful  Information  about  Risk 

We  now  make  a  few  brief  observations.  By  plotting  the  risk  curves  over  time  for 
each  strategy,  we  can  see  from  Figure  5  that  if  the  best  outcome  is  realized,  both  strategies 
reduce  risk  (as  seen  by  the  T1  curves  moving  left  and  becoming  more  vertical  than  the  start 
curves  over  most  of  the  range  of  analysis).  Likewise,  we  see  that  if  the  worst  outcome 
prevails  as  shown  in  Figure  6,  then  both  strategies  actually  increase  the  cost  risk  exposure 
of  the  project.  Graphing  risk  curves  over  time  thus  provides  a  metric  to  measure  the 
success  of  risk  mitigation  efforts. 
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Figure  5.  Risk  Exposure  Characteristics  for  a  Risk-seeking  Strategy  (Strategy  1)  and 
a  Risk-averse  Strategy  (Strategy  2),  Assuming  Good  Luck  Prevails  on  the  Project 
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Figure  6.  Risk  Exposure  Characteristics  for  a  Risk-seeking  Strategy  (Strategy  1)  and 
a  Risk-averse  Strategy  (Strategy  2),  Assuming  Murphy’s  Law  Prevails  on  the  Project 

But  the  graphs  provide  even  more  information.  They  allow  us  to  consider  the  risk 
mitigation  qualities  of  each  strategy  and  to  quantify  our  risk  exposure.  This  information  can 
be  used  to  choose  between  the  two  strategies.  Note  that  under  the  best-case  scenario 
(Figure  5),  at  the  start  of  the  project,  Strategy  1  offers  a  lower  risk  exposure  below  $1 ,500K, 
while  Strategy  2  offers  a  lower  risk  exposure  above  that  value.  Both  strategies  are  equal  in 
terms  of  exceedence  probability  (60  %)  at  the  “breakeven”  point  of  $1 ,500K.  What  do  we 
gain  by  extending  the  analysis  to  time  T1?  We  see  that  the  “breakeven”  point  is  lower 
($1 ,200K),  and  the  risk  at  that  point  is  also  lower  (40%).  So,  which  one  is  the  best  choice? 
If  we  were  optimists  and  certain  that  the  best  outcome  would  be  realized,  we  could  make  a 
choice  based  on  the  expected  benefits.  As  long  as  the  expected  benefits  of  the  RRA  are 
greater  than  $1 ,200,  we  would  choose  Strategy  2.  But  of  course,  we  have  no  such 
assurance,  so  let’s  examine  the  worst-case  scenario. 

Figure  6  shows  the  results  of  implementing  each  strategy  over  time  assuming  the 
worst  outcome  (Murphy’s  Law).  As  expected,  the  risk-seeking  Strategy  1  significantly 
increases  our  cost  risk  exposure  when  things  go  bad,  but  the  more  conservative  Strategy  2 
is  much  less  sensitive  to  bad  outcomes.  In  fact,  at  T1,  Strategy  2  dominates  Strategy  1 — 
meaning  it  has  a  lower  risk  for  any  value.  If  we  were  pessimists,  our  choice  would  be 
simple:  Strategy  2  is  especially  effective  in  providing  insurance  against  the  worst  outcomes. 

Which  strategy  is  chosen  depends  on  the  decision-maker’s  risk  aversion.  Is  he/she 
an  optimist  or  a  pessimist?  In  either  case,  if  the  expected  benefits  of  risk  mitigation  exceed 
$1 ,200K,  Strategy  2  is  the  best  choice.  We  believe  examining  risk  information  in  this  way 
provides  useful  insight  and  helps  project  managers  make  better  choices. 
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Table  1.  Strategy  1  Risk  Characteristics  at  Start  and  at  T1,  Assuming  that  Murphy’s 

Law  Prevails  on  the  Project 
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Table  2.  Strategy  2  Risk  Characteristics  at  Start  and  at  T1,  Assuming  that  Murphy’s 

Law  Prevails  on  the  Project 

Tables  1  and  2  suggest  that  the  use  of  mean  values  is  not  necessarily  a  cautious 
approach  for  planning  project  contingency. 
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Conclusions 

We  have  presented  a  method  for  evaluating  and  tracking  project-specific  risks  at  the 
microscopic  level.  This  type  of  analysis,  as  opposed  to  the  macroscopic-level  risk  analysis, 
is  essential  for  risk  management.  While  the  macro  level  provides  some  information  about 
total  cost  risk,  the  micro  level  allows  the  project  manager  to  plan  and  control  risk  response 
actions  that  influence  total  cost  risk. 

We  demonstrated  the  use  of  risk  decision  trees  to  model  the  evolution  of  the 
potential  RRAs,  and  we  used  risk  curves  to  evaluate  the  risk.  We  believe  risk  curves  are 
better  than  the  expected-value  results  usually  given  by  traditional  DT  analysis  because  they 
contain  all  the  risk  information  both  in  terms  of  probabilities  and  value  at  risk.  This  thorough 
approach  allows  management  to  consider  what  they  mean  by  “acceptable”  risk  and  explicitly 
models  the  tradeoff  between  risk  and  benefits  of  any  given  RRA. 

We  recommend  the  use  of  risk  curves  to  evaluate  the  performance  of  RRA  and  to 
track  their  performance  over  time.  If  the  RRA  is  working  (at  reducing  risk),  we  should  see 
the  corresponding  risk  curve  move  to  the  left  and/or  become  more  vertical.  This  tracking 
over  time  is  key  to  understanding  the  dynamic  nature  of  risk  management  and  can  reveal 
necessary  changes  in  strategy. 

Risk  curves  derived  from  Monte  Carlo  simulation  on  DTs  are  particularly  useful  when 
analysts  are  comparing  different  risk-mitigation  strategies.  The  “breakeven”  points  help  the 
risk  manager  understand  the  conditions  under  which  each  strategy  is  most  appropriate. 
Combined  with  scenario  analysis,  it  offers  an  opportunity  to  make  cost-benefit  tradeoffs 
among  strategies. 

We  think  that  these  results  provide  the  detailed  information  that  program  managers 
need  and  want  when  they  face  hard  decisions  on  programs.  There  is  a  cost  for  this  type  of 
analysis,  but  it  is  small  considering  the  potential  benefits.  The  proposed  approach  is  both 
practical  and  mathematically  valid  and  can  be  implemented  using  commercially  available 
tools  such  as  Crystal  Ball  and  @Risk.  The  challenge  is  to  start  implementing  these  more 
refined  cost  models  and  risk  management  practices. 
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Abstract 

This  paper  describes  how  the  US  Navy  structures  fixed-price  and  fixed-price, 
incentive-fee  shipbuilding  contracts  and  how  labor-  and  material-cost  indexes  can  mitigate 
shipbuilder  risk  in  either  type  of  contract.  The  Navy  frequently  uses  the  Steel  Vessel 
material-cost  index,  a  Bureau  of  Labor  Statistics-derived  cost  index  based  on  the  mix  of 
materials  in  a  typical  commercial  cargo  ship  constructed  in  the  1950s.  The  Steel  Vessel 
Index  has  excessive  weighting  on  iron  and  steel,  thereby  providing  shipbuilders  with  a 
mismatch  between  their  actual  and  the  Index-assumed  material-cost  structure.  We 
recommend  the  Navy  use  a  material-cost  index  with  more  up-to-date  weightings. 

Introduction 

The  Navy  wants  to  provide  its  shipbuilders  with  appropriate  incentives  to  produce 
militarily  effective  vessels  at  minimum  cost  to  the  Navy. 
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Fixed-price  contracts  provide  incentive  to  a  shipbuilder  to  produce  at  minimum  cost. 
After  contract  award,  cost  savings  the  shipbuilder  can  implement  flow  directly  to  the 
shipbuilder,  resulting  in  higher  profit.  Conversely,  cost  overruns  are  borne  by  the 
shipbuilder,  resulting  in  lower-than-anticipated  profits. 

Fixed-price  contracting  becomes  problematic,  however,  when  a  shipbuilder  is  forced 
to  bear  risk  outside  of  its  control.  For  instance,  ship  construction  requires  material  inputs 
like  steel,  wire,  cable,  and  myriad  others.  If  the  global  prices  of  these  commodities  rise,  a 
fixed-price  shipbuilder  will  have  lower  profits  (or  increased  losses)  external  to  the 
shipbuilder’s  efforts. 

Ultimately,  the  Navy  can  induce  a  shipbuilder  to  agree  to  any  arrangement,  including 
having  the  shipbuilder  bear  material-cost  risk,  by  offering  the  shipbuilder  a  high  enough 
price.  But  it  is  likely  to  be  preferable,  at  least  ex  ante,  for  the  Navy  to  dissipate  risk  external 
to  its  shipbuilder  in  order  to  pay  less  for  the  systems  the  Navy  needs. 

Conversely,  the  Navy  should  not  fully  immunize  a  shipbuilder  against  risks  within  the 
shipbuilder’s  control,  e.g.,  if  the  shipbuilder’s  own  failures  cause  a  cost  overrun.  In  such  a 
case,  the  shipbuilder  should  incur  at  least  a  portion  of  the  loss.  Of  course,  it  can  sometimes 
be  difficult  to  distinguish  problems  within  a  shipbuilder’s  control  versus  those  caused  or 
exacerbated  by  Navy  decisions  (e.g.,  changing  requirements)  versus  those  related  to 
external  issues  (e.g.,  the  rising  global  price  of  steel).  The  Navy  uses  labor-  and  material- 
cost  indexes  to  attempt  to  correct  for  several  significant  cost  risks  outside  its  shipbuilders’ 
control.  The  indexes  reflect  industry-  or  economy-wide  costs,  not  the  costs  of  the  specific 
shipbuilder. 

How  the  Navy  uses  Labor-  and  Material-cost  Indexes 

In  this  section  of  the  paper,  we  present  illustrative  examples  of  how  the  Navy  uses 
labor-  and  material-cost  indexes.  We  start  with  a  highly  oversimplified  example  of  a  fixed- 
price  contract  to  illustrate  the  basic  intuition.  Subsequently,  we  turn  to  an  enhanced  (though 
still  less  complex  than  reality)  example  of  a  contract  more  in  accord  with  current  Navy 
practices.  This  latter  example  is  a  Fixed-Price,  Incentive  Fee  (FPIF)  contract.  An  FPIF 
contract  is  no  longer  a  “pure”  fixed-price  contract  in  that  it  requires  the  Navy  and  the 
shipbuilder  to  share  cost  changes  from  the  negotiated  level  with  incentives  and  disincentives 
for  underruns  and  overruns  (whereas  a  textbook  fixed-price  contract  would  not).  The 
shipbuilder’s  actual  costs  are  considered  in  an  FPIF  contract;  they  are  not  in  a  fixed-price 
contract. 

A  Very  Simple  Example.  Let  us  suppose  the  Navy  signs  a  fixed-price  contract  for  a 
$220  million  ship  on  January  1, 2007,  with  completion  scheduled  for  January  1, 2010.  If 
$100  million  of  the  payment  is  to  cover  expected  labor  costs,  another  $100  million  is  to 
cover  expected  material  costs,  and  the  final  $20  million  is  intended  to  be  contractor  profit. 

Of  course,  the  actual  cost  the  shipbuilder  incurs  determines  the  shipbuilder’s  profit.  Figure  1 
shows  the  shipbuilder’s  profit  as  a  function  of  the  actual  labor  and  material  cost  of  the  ship. 
Increasing  costs  reduce  shipbuilder  profits  dollar-per-dollar. 
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Figure  1.  Shipbuilder  Profit  as  a  Function  of  Labor  and 
Material  Cost  with  a  Fixed-price  Contract 

Adding  material-cost  indexes  to  this  fixed-price  contract  would  protect  the  shipbuilder 
against  exogenous  cost  risk. 

Let  us  also  suppose,  during  the  period  2007-2010,  the  external  labor-cost  index 
designated  in  the  contract  goes  up  5%,  while  the  designated  material-cost  index  goes  up 
20%.  Then  the  Navy’s  actual  payment  to  its  shipbuilder  would  be  $245  million  ($105  million 
for  labor,  $120  million  for  materials,  $20  million  in  intended  or  target  profits — assuming  the 
profit  level  does  not  increase  with  the  indexes).  The  shipbuilder’s  actual  profit  would  then  go 
up  and  down  based  on  whether  their  actual  cost  growth  was  above  or  below  the  indexes’. 
Obviously,  it  is  of  central  importance  that  the  cost  indexes  are  agreed  upon  up  front. 

If,  on  the  other  hand,  the  labor-cost  index  had  risen  5%  while  the  material-cost  index 
had  fallen  10%,  the  Navy’s  payment  to  the  shipbuilder  would  be  $215  million  ($105  million  in 
labor,  $90  million  in  materials,  $20  million  for  target  profit).  Again,  actual  profit  would 
depend  on  whether  the  shipbuilder’s  total  costs  had  fallen  less  than  or  more  than  the 
indexes  suggested. 

Both  this  example  and  the  one  that  follows  are  over-simplified.  Both  examples 
assume  all  labor  is  incurred  and  material  purchased  on  the  last  day  of  the  contract.  If  one 
alternatively  assumes  the  postulated  inflation,  labor  hours,  and  material  purchases  occur 
uniformly  between  2007  and  2010,  the  average  inflation  rate  would  be  half  as  large.  In 
reality,  material  purchases  peak  before  labor  hours  are  incurred,  so  there  are  two  cost  timing 
distributions  to  account  for.  Actual  Navy  escalation  clauses  calculate  these  effects  on  actual 
costs  incurred  monthly.  The  Appendix  discusses  such  an  enhancement. 
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A  More  Realistic  Example.  The  Navy  does  not  generally  write  shipbuilding 
contracts  that  are  as  simple  as  the  preceding  example.  Instead,  the  norm  is  to  use  FPIF 
contracts  with  “compensation  adjustment  clauses”  or  “escalation  provisions”  to: 

■  Ensure  the  incentive  provision  operates  independent  of  outside  economic  forces  that 
impact  shipbuilder  costs. 

■  Keep  the  shipbuilder  from  including  contingent  amounts  in  its  price  to  cover 
economic  uncertainty  associated  with  external  cost  pressure. 

In  this  approach,  subsequent  changes  in  specified  cost  indexes  result  in  payments 
(or  refunds)  tied  to  the  shipbuilder’s  actual  labor  and  material  costs  incurred.  Notice  this 
approach  is  no  longer  a  “pure”  fixed-price  contract;  shipbuilders’  actual  costs  are 
considered.  FPIF  contracts  actually  operate  as  cost-type  incentive  contracts  within  a  certain 
range  of  costs. 

We  can  consider  a  similar  example  as  above  with  the  Navy  signing  a  contract  for  a 
ship  on  January  1, 2007,  with  completion  scheduled  for  January  1, 2010.  It  is  anticipated 
$100  million  will  be  spent  on  labor  and  another  $100  million  on  material.  Let  us  suppose  the 
Navy  also  agrees  to  a  10%  target  profit  rate  and  a  sharing  ratio  of  50/50  for  increases  or 
decreases  in  cost.  Figure  2  illustrates  shipbuilder  profit  under  this  FPIF  contract  versus  the 
preceding  fixed-price  case  (prior  to  consideration  of  cost-index  issues).  Since  this  FPIF 
contract  has  cost-change  sharing  between  the  Navy  and  the  shipbuilder,  the  FPIF  line  is 
flatter. 


Figure  2.  Shipbuilder  Profit  as  a  Function  of  Labor  And  Material  Cost 
with  Different  Contract  Structures 
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As  above,  it  would  enhance  realism  to  include  labor-  and  material-cost  indexes  into 
this  contract. 

Let  us  suppose,  during  the  period  2007-2010,  the  labor-cost  index  designated  in  the 
contract  goes  up  5%,  while  the  designated  material-cost  index  is  up  20%.  We  assume  base 
period  labor  and  material  costs  of  $1 00  million  each.  If  the  shipbuilder’s  actual  labor  cost 
was  $105  million,  the  Navy  would  pay  a  compensation  adjustment  of  $5  million  ((0.05 
divided  by  1 .05)  multiplied  by  $1 05  million).10  If  actual  material  costs  turned  out  to  be  $1 1 5 
million,  the  Navy  would  make  a  material  compensation  adjustment  of  $19.17  million  ((0.20 
divided  by  1 .20)  multiplied  by  $1 1 5  million).  The  “de-escalated  base  cost”  of  the  ship  would 
be  $195.83  million  (the  actual  $105  million  plus  $115  million  less  the  compensation 
adjustments  of  $5  million  and  $19.17  million).  The  $4.17  million  decrease  between  the  initial 
base  cost  and  the  de-escalated  base  cost  would  translate  into  a  $2.08  million  increase  in 
profit  for  the  shipbuilder  given  the  assumed  50/50  cost  change-sharing  ratio.  The 
shipbuilder  is  rewarded  because  actual  material  costs  did  not  rise  as  rapidly  (+15%)  as  the 
material-cost  index  (+20%). 

The  Navy’s  actual  payment  to  the  shipbuilder  would  be  comprised  of  $195.83  million 
in  de-escalated  base  cost,  $5  million  in  labor  escalation  payments,  $19.17  million  in  material 
escalation  payments,  $20  million  in  target  profit,  plus  $2.08  million  in  incentive  profit — 
totaling  $242.08  million.  Shipbuilder  profit  would  be  $22.08  million. 

By  contrast,  holding  the  shipbuilder’s  incurred  costs  the  same  as  above,  suppose  the 
labor-cost  index  had  again  risen  5%  while  the  material-cost  index  fell  10%.  The  labor 
compensation  adjustment  would  remain  $5  million  ((0.05  divided  by  1 .05)  multiplied  by  $105 
million).  The  material  compensation  adjustment  would  now  be  a  reimbursement  from  the 
shipbuilder  of  $12.78  million  ((-0.10  divided  by  0.90)  multiplied  by  $1 15  million).  The  “de- 
escalated  base  cost”  of  the  ship  would  be  $227.78  million  ($1 05  million  plus  $115  million 
minus  $5  million  plus  $12.78  million).  This  increase  in  the  de-escalated  base  cost  would 
result  in  a  $13.89  million  profit  penalty  for  the  shipbuilder  (50%  of  the  difference  between 
$227.78  million  and  $200  million).  Then,  the  Navy  would  pay  the  shipbuilder  $226.1 1  million 
($227.78  million  in  de-escalated  base  cost  plus  $5  million  in  labor  escalation  payments  less 
a  $12.78  million  material  de-escalation  reimbursement  plus  $20  million  in  target  profit  less  a 
$13.89  million  incentive  profit  penalty).  The  shipbuilder  profit  would  be  $6.1 1  million. 

As  in  the  “Very  Simple  Example,”  we  have  ignored  realistic  timing  issues,  e.g.,  the 
fact  that  median  material  cost  probably  precedes  the  median  labor  cost  and  that  neither  cost 
is  incurred,  on  average,  in  2010.  The  Appendix  discusses  the  effects  of  incorporating  labor 
and  material-cost  time-phasing. 

Figure  3  summarizes  the  differential  results  of  these  examples,  holding  fixed  that  the 
labor-cost  index  increased  5%,  while  realized  shipbuilder  costs  were  $115  million  for 
material  and  $105  million  for  labor.  Not  surprisingly,  when  the  shipbuilder  spends  more  on 
material  than  included  in  the  original  price  while  the  overall  material  market  has  falling 
prices,  the  cost  disincentive  built  into  the  contract  reduces  the  Navy’s  payment  and,  hence, 
the  shipbuilder’s  profit.  (The  shipbuilder  would  have  performed  very  poorly  if  it  paid  $1 1 5 
million  for  material  while  material  prices  were,  on  average,  falling.) 


10  For  expositional  simplicity,  we  are  assuming  actual  labor  costs  match  the  increase  in  the 
labor-cost  index,  allowing  us  to  concentrate  on  material-cost  issues. 
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Figure  3.  Shipbuilder  Profits  Are  Greater  When  the  Material-cost  Index  Rises  More, 

Realized  Costs  Held  Constant 

The  Fixed-price  Contract  line  and  the  FPIF  Contract  curve  cross  at  a  15%  increase 
in  the  material-cost  index.  We  have  assumed  the  shipbuilder’s  actual  material-cost  increase 
was  15%  or  $15  million.  If  the  shipbuilder  can  keep  its  actual  material-cost  growth  below  the 
index  level,  its  reward  is  greater  in  the  fixed-price  case,  in  which  there  is  no  cost-change 
sharing  with  the  Navy.  Conversely,  the  shipbuilder’s  profit  does  not  diminish  as  rapidly  if  its 
actual  material  costs  increase  more  than  the  Material-cost  Index  with  the  FPIF  contract’s 
cost  sharing. 

If  the  shipbuilder’s  skillful  management  kept  ship  costs  from  rising  as  much  as  similar 
costs  in  the  general  economy,  greater  profits  are  an  appropriate  reward.  Flowever,  if  greater 
profits  result  from  escalation  payments  calculated  by  an  external  price  index  that  does  not 
accurately  reflect  what  the  shipbuilder  purchases,  then  greater  profit  is  not  warranted. 
Conversely,  it  would  be  unfair  to  penalize  a  shipbuilder  if  an  inappropriate  cost  index 
declines  or  increases  less  than  the  shipbuilder’s  actual  cost  environment. 

The  Steel  Vessel  Index 

A  longtime  material-cost  index  in  Navy  shipbuilding  is  the  “Steel  Vessel  Index.” 

Based  on  an  estimate  by  the  Maritime  Administration  of  the  mix  of  materials  in  a  typical 
commercial  cargo  ship  constructed  in  the  1950s  (GAO,  1972),  it  is  a  weighted  average  of 
three  Bureau  of  Labor  Statistics  (BLS)  producer  price  indexes  (45%  Iron  &  Steel,  40% 
General  Purpose  Machinery  and  Equipment,  and  15%  Electrical  Machinery  and  Equipment). 
If,  for  instance,  the  Iron  &  Steel  price  index  increased  3%  in  a  year,  the  General  Purpose 
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Machinery  index  increased  2%,  and  the  Electrical  Machinery  index  fell  1%,  the  Steel  Vessel 
Index  would  increase  2%  (0.45*0.03+0.4*0.02-0.15*0.01 )." 

One  criticism  of  the  Steel  Vessel  cost  index  is  that  it  does  not  accurately  cover  the 
materials  used  in  building  a  modern  ship.12  No  modern  US  Navy  ship,  for  instance,  has  45% 
of  its  material  costs  in  Iron  &  Steel.  To  combat  this  shortcoming,  the  DDG-5113  and  T-AKE14 
programs  created  their  own  material-cost  indexes,  using  different  weights  on  the  same  three 
underlying  BLS  indexes  (DDG-51:  20%  Iron  &  Steel,  43%  General  Purpose  Machinery,  37% 
Electrical  Machinery;  T-AKE:  10%  Iron  &  Steel,  60%  General  Purpose  Machinery,  30% 
Electrical  Machinery).  See  Pfeiffer  (2006).  In  the  preceding  paragraph’s  example,  whereas 
the  Steel  Vessel  Index  would  increase  2%,  the  DDG-51  index  would  increase  1 .09% 
(0.2*0.03+0.43*0.02-0.37*0.01),  and  the  T-AKE  index  would  increase  1.2% 
(0.1*0.03+0.6*0.02-0.3*0.01). 

There  is  an  additional  challenge  with  any  of  these  indexes:  even  if  one  correctly 
identified  the  mix  of  materials  that  went  into  the  ship,  the  materials  would  be  purchased  at 
different  stages  of  ship  construction.  Steel,  for  instance,  is  required  early  in  the  construction 
process.  Conversely,  combat  systems  and  electrical  equipment  (perhaps  more  akin  to 
General  Purpose  or  Electrical  Machinery)  are  not  delivered  to  the  shipyard  and, 
consequently,  do  not  become  incurred  costs  until  much  later  in  construction.  Time-phasing 
the  mix  of  an  overall  material-cost  index  could  provide  greater  fidelity.  However,  it  is  unlikely 
any  material-cost  index  will  completely  dissipate  a  shipbuilder’s  exogenous  material-cost 
risk. 

Historically,  BLS’s  Iron  &  Steel  price  index  has  been  much  more  volatile  than  the 
General  Purpose  Machinery  or  Electrical  Machinery  indexes.  Figure  4  displays  these 
indexes’  quarterly  returns  (with  a  positive  “return”  if  the  cost  index  value  went  up,  negative  if 
it  fell)  between  the  second  quarter  of  1 94715  and  the  fourth  quarter  of  2006.  We  also  display 
the  quarterly  change  in  the  Bureau  of  Economic  Analysis’  (BEA)  Gross  Domestic  Product 
(GDP)  price  deflator,  a  measure  of  overall  inflation  in  the  economy. 


41There  does  not  appear  to  be  an  Air  Force  analog  to  the  Steel  Vessel  Index.  Air  Force  procurement 
contracts  may  include  BLS-based  labor-  or  material-cost  indexing,  but  this  is  done  on  a  case-by-case 
basis  at  the  discretion  of  the  program  office.  There  is  no  standard  aircraft  material-cost  index.  An 
aircraft’s  construction  duration  is  typically  much  briefer  than  that  of  a  ship,  so  inflation  issues  are  less 
prominent. 

12lndeed,  criticism  of  the  Steel  Vessel  Index  pre-dates  what  we  might  term  “modern”  ships.  Geismar 
(1975)  suggests  the  Steel  Vessel  Index  was  ill-suited  to  the  DD963,  Spruance  Class  destroyer,  and 
the  LHA,  Marine  amphibious  assault  ship — two  1970s-era  ship  programs.  (Both  of  these  ships  were 
very  late  in  delivering,  implying  inflation  issues  proved  to  be  more  important  than  would  have  been 
the  case  had  their  production  been  more  timely.) 

13The  DDG-51 ,  the  USS  Arleigh  Burke ,  is  a  destroyer  commissioned  on  July  4,  1991 .  The  moniker 
“DDG-51  ”  refers  to  the  class  of  destroyers  of  which  the  USS  Arleigh  Burke  was  the  first  (US  Navy, 
2006). 

14“T-AKE”  refers  to  the  Lewis  and  Clark  class  of  dry  cargo/ammunition  ships.  The  USNS  Lewis  and 
Clark,  the  USNS  Sacagawea,  and  the  USNS  Alan  Shepard  have  been  delivered  to  the  Navy;  the 
USNS  Richard  Byrd  is  under  construction  (US  Navy,  2007;  Bigelow,  2007). 

15Monthly  BLS  data  on  these  cost  indexes  are  available  back  to  January  1947.  However,  the  BEA 
GDP  deflator  data  are  only  available  quarterly,  so  we  aggregated  the  BLS  data  to  the  quarter  level. 
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Figure  4.  Quarterly  Changes  in  Different  Cost  Indexes 

(US  DoL,  BLS,  n.d.;  US  DoC,  BEA,  n.d.) 

Naturally,  given  the  Steel  Vessel  Index’s  greater  relative  weighting  of  the  Iron  &  Steel 
price  index,  it  has  been  more  volatile  than  the  DDG-51  or  T-AKE  indexes.  In  Figure  5,  we 
plot  the  standard  deviation  in  the  quarterly  return  and  the  mean  quarterly  return  for  the  three 
ship  material-cost  indexes  and  the  GDP  deflator  between  the  second  quarter  of  1947  and 
the  fourth  quarter  of  2006. 16  The  Steel  Vessel  Index  has  the  greatest  standard  deviation  in 
its  quarterly  return. 


16None  of  the  three  ship  material-cost  indexes  existed  in  1947.  But,  we  can  use  BLS  data  to 
retrospectively  compute  how  they  would  have  evolved. 
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Figure  5.  Quarterly  Standard  Deviation  and  Average  Return 
of  Different  Material-cost  Indexes 

What  Figure  5  does  not  show  is  how  closely  correlated  any  of  these  indexes  is  with 
the  actual  cost  variability  a  shipbuilder  experiences.  The  best  cost  index  is  the  one  that 
minimizes  a  shipbuilder’s  exogenous  risk  and,  therefore,  minimizes  the  risk  premium  the 
Navy  must  pay  the  shipbuilder.  We  know,  however,  the  Steel  Vessel  Index  over-represents 
iron  and  steel  costs  in  current  naval  warship  contracts. 

The  fact  the  Steel  Vessel  Index  has  had  a  mean  quarterly  return  greater  than  the 
other  indexes  and  greater  than  the  economy-wide  inflation  rate  is  not  prima  facie  bad  news 
for  the  Navy.  In  a  competitive  setting,  a  shipbuilder  will  submit  a  lower  bid  up  front  if  it 
expects  super-normal  escalation.  Therefore,  the  Navy’s  expected  costs  are  not,  in 
equilibrium,  affected  by  the  Index’s  mean. 

What  is  more  problematic  is  the  known  mismatch  between  the  Steel  Vessel  Index’s 
composition  and  a  shipbuilder’s  material-cost  structure.  The  shipbuilder  bears  a  risk,  for 
instance,  that  the  prices  of  iron  and  steel  may  tumble  while  the  shipbuilder’s  do  not.  A  risk- 
averse  shipbuilder  will  require  a  premium  to  bear  this  mismatch-driven  risk. 

This  mismatch-driven  risk  could  be  reduced  if  the  shipbuilder  could  take  a  short 
position  on  steel  futures,  i.e.,  hedge  against  the  risk  steel  prices  will  fall.  Currently,  however, 
there  is  no  functioning  steel  futures  market.17 


17There  is  an  ongoing  debate  as  to  the  feasibility  and  desirability  of  a  steel  futures  market. 
See,  for  instance,  Anderson  (2006). 
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Paradoxically,  if  the  shipbuilder  locked  in  its  steel  input  prices  through  a  long-term, 
fixed-price  contract  with  a  steel  mill,  the  shipbuilder’s  mismatch-driven  risk  could  be 
exacerbated,  not  mitigated.  If  future  steel  prices  fell,  the  shipbuilder  would  receive  no 
advantage  on  the  cost  side  while  receiving  reduced  revenue  from  the  Navy. 

We  do  not  know  the  “right”  material-cost  index  to  use  to  minimize  a  shipbuilder’s 
material-cost  risk.  We  do  know,  however,  the  Steel  Vessel  Index  is  imperfect  due  to  its 
over-representation  of  iron  and  steel.  As  shown  in  Figure  5,  there  is  little  difference  between 
the  DDG-51  and  T-AKE  approaches;  their  quarterly  returns  are  positively  correlated  at  the 
0.985  level.  (By  contrast,  the  Steel  Vessel  index  has  a  0.936  correlation  with  the  DDG-51 
index  and  0.873  with  T-AKE.) 

Of  the  three  Navy  material-cost  indexes,  T-AKE  (0.655)  and  DDG-51  (0.636)  are 
more  highly  correlated  with  the  GDP  deflator  than  is  the  Steel  Vessel  Index  (0.538).  The 
explanation  for  the  Steel  Vessel  Index’s  relative  lack  of  correlation  with  overall  inflation  in  the 
economy  is  that  the  Iron  &  Steel  cost  index  has  a  much  lower  correlation  (0.360)  with  the 
GDP  deflator  than  the  General  Purpose  Machinery  (0.634)  and  Electrical  Machinery  (0.609) 
cost  indexes.  So,  a  material-cost  index  that  over-samples  Iron  &  Steel  moves  away  from 
representation  of  economy-wide  costs. 

The  foremost  argument  in  favor  of  the  Steel  Vessel  Index  is  its  familiarity  and, 
consequently,  the  comfort  some  shipbuilders  have  with  the  Index.  Almost  everyone  we  met 
in  the  nautical  construction  industry  knows  of  the  Steel  Vessel  Index,  and  most  have 
experience  with  contracts  tied  to  it.  The  Steel  Vessel  Index  is,  perhaps,  akin  to  the  Dow 
Jones  Industrial  Average  in  that  one  would  not  invent  it  anew  (or  at  least  not  with  its  current 
weightings),  but  its  fame  and  tradition  keep  it  in  use.18 

If  shipbuilders  are  familiar  and  comfortable  with  the  Index,  the  Navy  and  the 
government  benefit,  as  this  may  imply  shipbuilders  can  be  paid  less  when  the  Index  is  in 
use.  The  best  material-cost  index  minimizes  the  exogenous  risk  shipbuilders  perceive  they 
face  so  as  to  therefore  minimize  Navy  ship  acquisition  costs.  Unless  one  believes  familiarity 
is  extremely  important,  however,  the  manifest  cost  structure  mismatch  of  the  Steel  Vessel 
Index  suggests  its  usage  does  not  minimize  the  Navy’s  expected  costs. 


Conclusions 

We  do  not  think  the  Navy  should  use  the  Steel  Vessel  Index  to  adjust  for  material- 
cost  changes  in  future  shipbuilding  contracts.  The  Steel  Vessel  Index  clearly  puts  excessive 
weight  on  Iron  &  Steel  relative  to  the  materials  actually  used  in  constructing  a  modern  ship. 
Usage  of  the  Steel  Vessel  Index  does  not  appropriately  mitigate  contractor  material-cost 


18Discussing  an  earlier  version  of  this  paper,  Jim  Jondrow  of  the  Center  for  Naval  Analyses  raised  the 
following  analogy  to  the  Navy’s  continued  use  of  the  Steel  Vessel  Index:  let  us  suppose  one  owned  a 
portfolio  that  mirrored  the  NASDAQ  Composite  Index,  but  one  observed  the  Dow  Jones  Industrial 
Average  (or  vice  versa).  On  March  10,  2000,  the  NASDAQ  Composite  Index  closed  at  an  all-time 
high  of  5046,  but  then  fell  precipitously,  ultimately  hitting  a  bottom  of  1 1 14  on  October  9,  2002.  See 
“Nasdaq  Composite”  (n.d.).  Meanwhile,  the  Dow  Jones  Industrial  Average  closed  at  9929  on  March 
10,  2000,  and  at  7286  on  October  9,  2002.  See  Yahoo!  Finance  (n.d.).  The  indexes  were  positively 
correlated  with  one  another,  but  the  magnitudes  of  the  changes  were  sharply  different. 
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risk.  Indeed,  from  a  shipbuilder’s  perspective,  a  new  risk  is  created:  the  risk  the  prices  of 
what  the  shipbuilder  actually  buys  will  rise  faster  than  the  price  of  steel. 


The  shortcomings  of  the  Steel  Vessel  Index  have  been  known  for  many  years.  The 
DDG-51  and  T-AKE  programs  created  their  own  material-cost  indexes  with  lower  weight  on 
Iron  &  Steel.  Their  material-cost  indexes,  which  empirically  have  been  highly  correlated  with 
one  another,  are  doubtlessly  better  indexes  than  the  Steel  Vessel  Index,  though  they  still 
appear  to  put  too  much  weight  on  Iron  &  Steel  (DDG-51:  20%,  T-AKE:  10%). 

We  urge  the  Navy  to  develop  a  “Modern  Vessel  Index”  that  more  appropriately 
represents  the  material  used  in  constructing  ships.  Movement  toward  a  better  index  would 
also  be  an  opportunity  to  explore  a  time-phased  material-cost  index — e.g.,  reflect  the  fact 
shipbuilders  typically  buy  keel  steel  early  in  production,  with  on-board  electronics  procured 
much  later  in  the  construction  process.  The  more  accurately  a  material-cost  index  captures 
a  shipbuilder’s  external  material-cost  risk,  the  less  the  Navy  may  expect  to  pay  its 
shipbuilders. 
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Appendix  1.  Time-phasing  Material-  and  Labor-cost  Indexes 

In  the  examples  in  the  body  of  this  paper,  we  unrealistically  assume  all  shipbuilder 
expenses  are  borne  at  the  end  of  the  three-year  build  cycle;  we  then  use  the  material  and 
labor-cost  index  values  at  the  end  of  the  build  cycle  to  determine  the  Navy’s  payment  to  the 
shipbuilder. 

In  fact,  actual  Navy  shipbuilding  contracts  are  more  sophisticated.  Instead  of 
assuming  all  costs  are  incurred  at  the  end  of  the  build  cycle,  a  month-by-month  expenditure 
pattern  is  assumed,  an  illustrative  example  of  which  is  presented  in  Figure  A.1. 


■4—  Material 
m —  Labor 


Build  Cycle  Month 


Figure  A.1.  An  Illustration  of  Assumed  Percentages  of  Total  Costs  Incurred  by  Month 

Figure  A.1,  like  most  Navy  shipbuilding  contracts,  assumes  the  shipbuilder  generally 
bears  material  costs  (e.g.,  buying  keel  steel)  in  front  of  labor  costs.19 

The  effect  of  this  cost  time-phasing  assumption  is  to  move  forward  the  implicit 
median  date  of  contractor  expenditure  and,  therefore,  to  reduce  (assuming  the  labor  and 
material-cost  indexes  generally  increase)  the  shipbuilder’s  inflation-related  adjustment.  This 
reduction  is  generally  more  marked  for  material  costs  because  of  the  standard  assumption 
material  costs  are  borne  sooner. 


19  Standard  shipbuilding  contracts  do  not,  however,  differentiate  between  types  of  material.  An 
enhancement  we  urge  the  Navy  to  consider  would  be  to  break  up  material  costs,  e.g.,  assume  steel 
expenditures  for  the  keel  precede  electronics-type  expenditures  for  onboard  weapon  systems. 
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Revisiting  a  Very  Simple  Example.  As  above,  let  us  suppose  the  Navy  signs  a 
fixed-price  contract  for  a  ship  on  January  1 , 2007,  with  completion  scheduled  for  January  1 , 
2010.  We  assume  the  ship  has  $100  million  each  in  expected  labor  and  material  costs  plus 
an  additional  expected  or  target  profit  of  $20  million.  However,  labor  and  material  costs  are 
expected  to  be  borne  in  accord  with  Figure  A.I’s  pattern. 

Let  us  suppose,  during  the  period  2007-2010,  the  external  labor-cost  index 
designated  in  the  contract  goes  up  5%  while  the  designated  material-cost  index  goes  up 
20%.  In  addition,  (though  one  need  not  make  this  pedagogically  simplifying  assumption) 
those  increases  occur  uniformly  over  the  36-month  build  period.  Then  the  effective  increase 
in  assumed  labor  costs  (given  Figure  A.I’s  cost  incurrence  pattern)  is  2.6%,  while  the 
increase  in  material  costs  is  8.1%.  Notice  the  effective  increase  in  labor  costs  is  52%  of  the 
3-year  total  increase,  while  the  effective  increase  in  material  costs  is  40%  of  the  3-year  total 
increase;  this  differential  reflects  the  assumption  that  material  costs  generally  precede  labor 
costs. 

In  the  “Very  Simple  Example’s”  contract,  the  Navy’s  actual  payment  to  the 
shipbuilder  would  be  $230.7  million  ($102.6  million  for  labor,  $108.1  million  for  material,  $20 
million  for  target  profit). 

Time-phasing  contracts  does  not  axiomatically  imply  reduced  shipbuilder  profits 
(though  one  might  draw  such  an  inference  from  juxtaposing  this  example  to  the  body  of  the 
paper’s  “Very  Simple  Example”).  The  shipbuilder’s  initial  bid  will  be  made  cognizant  of  how 
(and  whether)  labor  and  material  costs  are  to  be  indexed.  A  less  generous  (but  more 
accurate)  indexing  approach  of  this  sort  will  doubtlessly  cause  the  shipbuilder’s  bid  to  be 
greater. 

Revisiting  a  More  Realistic  Example.  In  our  “More  Realistic  Example,”  the  Navy 
provided  the  shipbuilder  with  an  FPIF  contract  with  a  50/50  sharing  ratio  on  increases  or 
decreases  in  costs. 

As  noted  above,  if  the  labor-cost  index  designated  in  the  contract  goes  up  5%  in 
three  years,  while  the  designated  material-cost  index  goes  up  20%.  The  effective  increases 
in  the  indexes  are  2.6%  and  8.1%,  respectively,  adjusting  for  Figure  A.I’s  assumed 
expenditure  pattern. 

In  “A  More  Realistic  Example,”  we  had  actual  labor  costs  of  $105  million.  If  we 
scaled  this  value  down  in  accordance  with  Figure  A.1,  the  “adjusted”  actual  labor  costs 
would  be  $102.6  million.  Similarly,  “adjusted”  actual  material  costs  would  be  $106.1  million. 

The  labor  compensation  adjustment  would  now  be  $2.6  million  ((0.026  divided  by 
1 .026)  multiplied  by  $102.6  million).  The  material  cost  adjustment  would  be  $8.0  million 
((0.081  divided  by  1.081)  multiplied  by  $106.1  million).  The  de-escalated  base  costs  of  the 
ship  would  be  $198.1  million  (the  “adjusted”  actual  $102.6  million  in  labor  and  $106.1  million 
in  material  less  the  compensation  adjustments  of  $2.6  million  for  labor  and  $8.0  million  for 
material).  The  shipbuilder  profit  would  be  increased  by  $0.9  million. 

As  in  the  body  of  the  paper,  the  shipbuilder’s  profit  is  greater,  holding  its  actual 
incurred  costs  constant,  when  the  material-cost  index  grows  more.  The  effect  of  time¬ 
phasing  is  to  roughly  halve  (more  of  a  reduction  for  material  than  for  labor)  the  measured 
indexed  inflation  rate.  But  the  comparative  static  result  that  the  shipbuilder  is  better  off 
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when  the  material-cost  index  rises  more,  holding  costs  constant,  remains.  Again,  such 
rewards  are  appropriate  if  shipbuilder  management  held  costs  down  better  than  might  have 
been  expected.  Conversely,  if  greater  profits  were  received  because  an  index  used  to 
calculate  escalation  payments  is  flawed,  unwarranted  profits  may  result. 
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Kingdom 

Discussants: 

Peter  Tatham,  Commodore  (ret.),  Defence  Acquisition  Management 
Deputy  Course  Director,  Cranfield  University,  Defence  Academy  of  the 
United  Kingdom 

Dr.  David  M.  Moore,  Head  of  the  Acquisition  and  Logistics  Unit,  Cranfield 
University,  Defence  Academy  of  the  United  Kingdom 

Paper: 

Five  Key  Changes  for  the  Management  of  UK  Defence — An  Agenda  for 
Research? 

Peter  Tatham,  Commodore  (ret.),  Defence  Acquisition  Management 
Deputy  Course  Director,  Cranfield  University,  Defence  Academy  of  the 
United  Kingdom  and  Dr.  David  M.  Moore,  Head  of  the  Acquisition  and 
Logistics  Unit,  Cranfield  University,  Defence  Academy  of  the  United 
Kingdom 


Chair:  Dr.  Trevor  Taylor  is  Head  of  the  Department  of  Defence  Management  and  Security  Analysis 
at  Cranfield  University 's  faculty  at  the  Defence  Academy  of  the  UK.  He  was  previously  Professor  of 
International  Relations  at  Staffordshire  University,  and  between  1990  and  1993  was  Head  of  the 
International  Security  Programme  at  the  Royal  Institute  of  International  Affairs  (Chatham  House)  in 
London.  He  is  a  past  Chairman  of  the  British  International  Studies  Association,  and  has  been  Visiting 
Professor  at  the  National  Defence  Academy  in  Tokyo. 

Taylor  takes  a  close  interest  in  the  UK  Smart  Acquisition  initiative,  and  teaches  defence  acquisition 
on  his  Department's  Master  in  Defence  Administration,  Defence  Logistics  Management  and  Defence 
Acquisition  Management  degrees.  He  contributes  also  to  the  seven-week  and  two-week  courses  on 
the  Management  and  Governance  of  Defence  that  support  the  UK's  Conflict  Prevention  initiatives. 

In  1999,  he  facilitated  the  initial  short  courses  in  Smart  Acquisition  training  for  staff  of  the  MoD's 
Equipment  Capability  Customer,  and  he  was  appointed  in  the  summer  of  2000  as  the  leader  of  the 
Non-Advocate  Review  Team  for  Project  Bowman,  a  peer  review  group  supporting  the  IPT  Leader  and 
reporting  to  the  (former)  Equipment  Approvals  Committee.  He  was  elected  to  the  Council  of  the 
Defence  Manufacturers'  Association  in  2001  and  re-elected  in  2004.  He  was  appointed  as  a  member 
of  the  Markets  sub-group  of  the  Department  of  Trade  and  Industry  Aerospace  Innovation  &  Growth 
team  in  September  2002.  In  2006,  he  formed  part  of  the  industrial  consultation  panel  for  the  MoD's 
Sustained  Armoured  Vehicle  Capability  Pathfinder  group,  which  was  set  up  as  part  of  the  British 
Defence  Industrial  Strategy  document  of  2005. 
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He  has  published  extensively  on  European  security  and  defence  industrial  issues.  His  article  entitled 
“The  Delineation  of  Defense  Equipment  Projects  in  the  UK  Ministry  of  Defense”  was  published  in 
Defense  and  Security  Analysis  in  June  2004  (20(2));  his  article  “Governments  and  Industry”  was 
published  in  RUSI  Defense  Systems,  Summer  2004  (7(1));  and  his  article  “The  Place  of  Management 
in  Defence  (and  Defence  Education)”  was  published  in  the  RUSI  Journal  April  2005  (150(2),  24-28). 

Other  recent  publications  and  presentations  include  chapters  on  “The  Functions  of  a  Defence 
Ministry”  and  “Defence  Acquisition”  in  L.  Cleary  &  T.  McConville  (eds.),  Managing  Defence  in  a 
Democracy  (London:  Routledge,  2006);  “Globalisation  in  the  Defence  Industry”  in  Defence  and  Peace 
Economics  (12(3/4));  joint  editorship  of  A  Comparison  of  the  Defense  Acquisition  Systems  of  France, 
Great  Britain,  Germany  and  the  United  States  (VA:  Defense  Systems  Management  College  Press, 
1999);  “Europe's  Revolution  in  Defence  Industrial  Affairs,”  in  R.  Matthews  (ed.),  Managing  the 
Revolution  in  Military  Affairs  (London:  Palgrave,  2001);  and  “Arms  Procurement”  in  J.  Howarth  &  A. 
Menon  (eds.),  The  European  Union  and  National  Defence  Policy  (London:  Routledge,  1997).  His 
writing  on  Smart  Procurement  includes  pieces  in  the  RUSI  Journal  in  April  1998  (143(2))  and  June 
1999  (144(3))  and  an  article,  “Smart  Procurement  and  the  Aerospace  Sector,”  in  the  International 
Journal  of  Aerospace  Management,  June  2001  (1(2)). 

Discussant:  Peter  Tatham  joined  the  Royal  Navy  in  1970  and  served  in  a  variety  of  appointments 
during  his  career  of  some  35  years.  Highlights  include  Logistics  Officer  of  the  Aircraft  Carrier  HMS 
INVINCIBLE  in  1994/5  during  Operations  in  Bosnia  against  the  Former  Republic  of  Yugoslavia  and 
Chief  Staff  Officer  responsible  for  all  high-level  Personnel  and  Logistics  issues  emanating  from  the 
10,000  sailors  and  30  surface  ships  in  the  Royal  Navy  (1999-2000).  His  final  three  years  in  the 
Service  were  spent  in  the  Defence  Logistics  Organisation,  where  he  was  responsible  for  key 
elements  of  the  internal  programme  of  Change  Management  (2000-2004).  During  this  period,  he  also 
gained  an  MSc  in  Defence  Logistic  Management.  Following  his  retirement  from  the  RN,  he  joined  the 
staff  of  Cranfield  University,  where  he  lectures  on  Defence  and  Humanitarian  Logistics.  He  is  also 
carrying  out  his  Doctoral  research  into  the  issues  surrounding  the  role  of  shared  values  within  military 
supply  networks. 

Discussant:  David  Moore  worked  in  purchasing,  logistics  and  supply  chain  management  within 
public  sector  and  commercial  organisations  before  entering  academia.  With  the  University  of 
Glamorgan,  he  developed  and  delivered  Chartered  Institute  of  Purchasing  and  Supply  (CIPS) 
courses  for  organisations  such  as  British  Airways,  London  Underground  and  the  Civil  Service 
College.  These  were  followed  by  the  development  and  leadership  of  the  MBA  full-time  and  part-time 
programs,  as  well  as  the  MBA  “by  Directed  Learning  in  Bahrain.”  In  1996,  he  joined  Cranfield 
University,  where  he  initiated  the  BSc  (Hons)  Management  and  Logistics  course  and  then  the  MSc 
Defence  Logistics  Management.  Most  recently,  he  designed,  developed  and  delivered  the  MSc 
Defence  Acquisition  Management  course  for  which  he  is  now  Course  Director.  He  has  undertaken 
extensive  education,  training,  and  consultancy  assignments  in  the  UK,  USA,  Europe,  Middle  East  and 
Far  East  for  organisations  such  as  Shell,  British  Gas,  NHS,  Welsh  Office,  Coca  Cola,  Qatar  Steel  and 
the  Arab  Ship  Repair  Yard.  He  has  also  spoken  at  conferences  around  the  globe.  Particular  interests 
include  outsourcing,  the  use  of  contractors  for  service  provision,  and  developing  effectiveness 
through  professionalism.  He  has  written  a  number  of  books  and  book  chapters  plus  conference  and 
journal  papers.  Moore  served  in  the  RLC  (TA)  until  1999,  and  held  both  staff  and  CO  appointments  as 
Lt  Colonel.  His  doctoral  research  was  focused  upon  “Knowledge  as  the  Basis  of  Professionalism  in 
Procurement,  Logistics  and  Supply  Chain  Management.”  He  has  been  interested  for  some  time  in  the 
commercialised  approaches  to  logistics  and  their  application  in  humanitarian  aid  scenarios,  and 
especially  the  use  of  military  logistics  solutions  within  such  situations. 
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Abstract 

This  paper  is  conceptual  in  nature  and  reviews  five  of  the  key  management 
challenges  facing  those  charged  with  the  management  of  the  UK  Defence.  It  is  argued  that 
similar  challenges  face  many  Western  governments,  albeit  the  solutions  are  likely  to  be 
country-specific.  Given  the  paucity  of  academic  research  into  the  general  area  of  defence 
management,  it  is  suggested  that  there  is  considerable  potential  for  focussed  application  of 
ideas  and  concepts  from  a  broad  range  of  disciplines  that  will  help  improve  the  UK’s  ability 
to  maintain  its  peacetime  military  capability  efficiently  whilst  retaining  its  capacity  to  conduct 
operations  effectively.  Whilst  the  paper  does  not  seek  to  recommend  solutions  for  the 
issues  identified,  it  does  seek  to  expose  the  essential  features  as  a  means  of  broadening 
the  understanding  of  the  nature  of  the  challenge — and,  hence,  to  help  shape  the  research 
agenda. 

Introduction 

By  any  dispassionate  measure,  the  management  of  a  major  state’s  defence  needs  is 
a  hugely  complex  challenge.  People,  training,  equipment,  information,  infrastructure  and 
other  resources  have  to  be  integrated  to  generate  the  required  operational  capability.  Some 
spending  makes  an  immediate  and  direct  contribution  to  capability,  while  other 
investments — most  notably  in  defence  research  and  development — may  not  produce  usable 
assets  for  a  decade  or  more  (Taylor,  2005).  In  a  basic  sense,  the  outputs  of  defence 
spending  can  be  expressed  in  terms  of  units  of  forces  at  specified  rates  of  readiness  (with 
concomitant  levels  of  manning,  equipment,  individual  and  collective  training),  but  this 
disguises  the  range  of  missions  for  which  forces  may  be  expected  to  be  ready.  The  then- 
British  Prime  Minister  Tony  Blair  (2007)  and  General  Sir  John  Kiszely  (2006)  have  pointed 
out  the  challenges  of  maintaining  forces  that  are  ready  both  for  high-intensity  combat 
operations  and  for  peace-support  activities.  In  practice,  things  are  even  more  complex — 
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within  the  area  of  UK  defence  planning  assumptions,  there  is  a  seven-layer  taxonomy  of 
military  operations  from  Deliberate  Intervention  to  Evacuation  of  Non-combatants.  Even 
more  importantly,  there  are,  increasingly,  no  hard  and  fast  geographic  or  temporal  divisions 
in  the  operational  environment.  Hence,  as  observed  by  the  recently  retired  NATO  Deputy 
Supreme  Allied  Commander  for  Transformation,  we  have  the  development  of  what  is 
becoming  known  as  the  “three  block  war.”  “In  both  theatres  [Iraq  &  Afghanistan]  we  have 
had  high  tempo  warfighting  taking  place  in  the  same  50km  square  and  at  the  same  time  as 
peacekeeping  and  humanitarian  aid  operations”  (Forbes,  2004). 

The  aim  of  this  paper  is,  based  on  the  authors’  extensive  experience  of  studying  and 
implementing  defence  management,  to  consider  some  of  the  key  managerial  challenges 
that  face  those  who  are  charged  with  planning,  directing,  organising,  coordinating, 
monitoring  and  improving  the  use  of  the  resources  directed  to  the  defence  sector.  The 
paper  will  not  seek  to  identify  any  assured  route  to  success;  rather,  given  that  a  prior 
condition  for  any  such  prescription  is  awareness  of  the  pitfalls,  dilemmas  and  issues  that 
need  to  be  addressed,  this  paper  offers  a  view  of  five  of  the  most  important  challenges  that 
will  be  faced  by  those  charged  with  the  higher  management  of  defence.  It  has  a  UK  focus 
and  illustrates  the  challenges  identified  with  British  examples,  but  the  dilemmas  it  identifies 
are  of  universal  application.  The  paper  is,  therefore,  of  interest  to  a  wider  audience  than  just 
those  in  the  UK. 

Any  consideration  of  defence  management  must  take  into  account  contextual  factors 
in  addition  to  the  operational  and  capability  issues  noted  above.  In  the  UK  (and  in  many 
other  states)  some  important  elements  are: 

The  increasing  complexity  and  capability  of  military  platforms,  equipment  and 
information  systems  that,  while  bringing  clear  benefit  in  the  actual  prosecution  of 
warfighting,  involves  extensive  acquisition  and  support  costs.  These  have  been 
estimated  by  a  recent  authoritative  report  to  account  for  40%  of  the  UK  defence 
budget  (MOD,  2006,  June) — a  figure  that  could  rise  by  as  much  as  20%  if  it  were  to 
include  the  costs  of  the  engineering  and  support  Regiments  within  the  field  army,  etc. 
It  is  unsurprising,  therefore,  that  three  of  the  five  challenges  considered  by  the 
authors  lie  in  the  acquisition  and  support  arena. 

Shrinking  budgets  and  reducing  “headcount”  for  the  Armed  Forces  (and  their  civilian 
support  staff).  For  example,  the  manpower  strength  of  the  Royal  Navy  has  reduced, 
on  average,  by  2.25%  year  on  year  since  1950.  In  addition,  the  recent  “Gershon” 
reforms  require  a  reduction  of  1 1 ,340  Civil  Servants  from  the  UK  MOD  by  2008 
(MOD,  2006,  p.  36). 

An  increasingly  critical  public  which,  whilst  profoundly  supportive  of  the  individual 
soldier,  sailor  and  airman,  is  unconvinced  that  the  underpinning  managerial  structures  are 
“fit  for  purpose.”  This  was  underlined  by  the  survey  of  external  opinion  reported  in  the  MOD 
Annual  Report  and  Accounts  for  2006/7  (MOD,  2007,  Table  30) — in  which  those  indicating  a 
“favourable  impression”  of  the  Armed  Forces  was  76%,  whereas  those  with  a  “favourable 
impression”  of  the  MOD  was  just  under  half  at  44%. 

Given  the  breadth  and  depth  of  the  management  challenges  in  defence,  it  is 
surprising  that,  unlike  the  study  of  commercial  management  that  has  expanded  rapidly  since 
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the  end  of  the  Second  World  War,  defence  gets  comparatively  little  attention  from  scholars.20 
This  lack  of  academic  interest  in  the  management  of  defence  may  simply  be  because  it  is 
significantly  different  from  everything  else  or  because  it  requires  too  much  specialistic 
background  knowledge  for  the  generic  management  expert  to  make  sense  of  it  without 
significant  effort.  Therefore,  this  paper,  with  the  five  challenges  discussed  below,  is  an  effort 
to  conceptualise  some  important  defence  dilemmas  and  choices. 

Challenge  1 — Empowerment  versus  Coherence 

Striking  the  right  balance  between  achieving  conformity  without  stifling  potentially 
beneficial  individualism  is  a  major  challenge  in  many  areas  of  defence  management.  In 
wider  (and  mainly  commercial)  management  thought,  the  empowering  of  individuals  is  seen 
as  a  key  component  of  organisational  success.  As  explained  by  Senge  (1992,  pp.  287-288), 
“localness  means  unleashing  people’s  commitment  by  giving  them  the  freedom  to  act,  to  try 
out  their  own  idea  and  be  responsible  for  producing  results.”  He  also  added  that,  “localness 
is  especially  vital  in  times  of  rapid  change,”  when  people  lower  down  the  organisation  need 
a  clearer  sense  of  what  is  happening  to  them  and  how  they  can  best  respond.  Similarly, 
Cole  (2004,  pp.  201-202)  observes  that,  “the  best  practice  is  to  be  found  in  organisations 
that  use  delegation  positively  as  an  important  employee  motivator  as  well  as  a  means  of 
facilitating  effective  decision-making  throughout  the  enterprise.” 

In  the  operational  military  context,  British  Defence  Doctrine21  emphasises  the 
concept  of  “Mission  Command,”  in  which  the  high-level  commander’s  Strategic  Intent  is 
clearly  spelled  out.  Subordinates  are  then  encouraged  and  empowered  to  implement  this 
objective  as  they  see  fit  under  the  emerging  operational  circumstances. 

A  similar  theoretical  approach  has  been  adopted  in  the  UK’s  Smart  Acquisition22  (SA) 
programme,  which  is  designed — against  the  background  of  some  800  projects  which  are 
being  developed  through  around  100  Integrated  Project  Teams  (IPTs) — to  ensure  the 
introduction  and  support  of  defence  equipment  “Faster,  Cheaper,  Better  and  More 
Effectively  Integrated.”  The  underlying  SA  philosophy  was  designed  to  “empower”  the  IPT 
Leaders  (and,  indeed,  to  judge  them  on  their  ability)  to  develop  radical  solutions  to  the 
delivery  of  military  effect  within  a  prescribed  Performance,  Cost  and  Time  envelope. 


20  As  an  illustration,  an  examination  of  five  major  academic  journals  covering  logistics  and 
supply  chain  management  showed  that  out  of  1020  articles  published  between  January 
2000  and  December  2007,  only  one  discussed  issues  from  a  defence  perspective. 

21  Doctrine  is  defined  as  “the  best  estimate  of  the  way  the  UK’s  Armed  Forces  [. . .]  should 
go  about  their  military  business”  (MOD,  2001,  p.  3-1).  However,  as  eloquently  observed  by 
the  eminent  historian  Sir  Michael  Howard  (1974,  p.  7),  the  MOD  cannot  conduct 
experimentation  to  prove  theories  of  warfare  in  the  same  way  as  a  scientist;  thus,  inevitably, 
Doctrine  represents  an  “educated  guess.”  He  goes  on  to  suggest  that  success  will  favour 
the  military  force  that  best  develops  “the  capacity  to  adapt  oneself  to  the  utterly 
unpredictable,  the  entirely  unknown.” 

22  When  originally  launched  in  1999,  the  programme  was  called  “Smart  Procurement.”  However,  its 
title  was  changed  in  2002  to  “Smart  Acquisition”  as  a  means  of  better  reflecting  the  whole-life 
implications  of  procurement  decisions. 
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Unfortunately,  this  focus  on  the  achievement  of  a  successful  outcome  to  an 
individual  project  means  that  IPT  Leaders  can  be  inherently  reluctant  to  spend  their  budgets 
on  managing  important  interfaces  with  other  projects.  It  is  extremely  tempting  to  conclude 
that  they  should  always  be  sorted  out  by  the  IPT  responsible  for  the  other  side  of  the 
boundary!  Furthermore,  attempts  to  constrain  IPT-level  solutions  in  order  to  ensure  that 
they  allow  a  seamless  interface  with  other  IPTs  (and  virtual  constructs  such  as  the  Defence 
Supply  Chain)  frequently  result  in  an  increase  in  the  cost  of  the  proposed  solution. 

The  more  complex23  the  battlefield  becomes,  the  more  prominent  such  project 
interfaces  clearly  become.  For  example,  in  the  Army  environment,  a  battle  group  will  consist 
of  the  appropriate  mixture  of  infantry,  armour,  and  artillery  to  meet  the  threat  of  the  moment; 
but  the  grouping  is  likely  to  be  transient  according  to  the  nature  of  the  threat  faced. 
Integrating  the  logistic  support  for  these  so-called  “Agile  Mission  Groups”  in  such  a  way  that 
it  does  not  constrain  the  commander’s  freedom  drives  towards  a  unitary  support  solution  for 
the  operation  as  a  whole — but  this  is  in  direct  opposition  to  the  Smart  Acquisition  business 
drive  to  deliver  unique,  optimised  support  targeted  at  a  particular  platform  or  equipment. 

It  is  also  becoming  increasingly  apparent  that  the  integration  problems  associated 
with  information  systems  are  even  more  challenging,  not  least  as  the  technology  refresh  rate 
here  is  measured  in  terms  of  2-3  years  (rather  than  the  decades  for,  say,  the  transition  from 
steam  to  gas  turbine  as  a  means  of  maritime  propulsion).  The  challenge  of  integrating 
Information  Systems  is  also  exacerbated  by  both  the  internal  demands  of  the  UK’s  Network 
Enabled  Capability24  (NEC)  and  the  desire  to  link,  on  the  one  hand,  to  US  systems  and,  on 
the  other,  to  those  of  our  European  allies. 

In  order  to  resolve  this  conundrum,  a  number  of  concepts  have  been  championed. 
For  example,  some  look  to  the  expert  use  of  systems  engineering  techniques  (see  Stevens, 
Jackson,  Brook  &  Arnold,  1998);  others  suggest  that  a  central  guidance  and  lobbying  body 
such  as  the  Integration  Authority  within  the  UK’s  Defence  Equipment  and  Support  (DE&S) 
organisation  can  make  the  required  impact.  A  further  alternative  prescribes  defining  “open 
system  architectures”  to  allow  easy  integration  and  modification  of  modules — but  none  of 
these  approaches  has  yet  to  prove  entirely  satisfactory. 

Nevertheless,  one  procurement  choice  that  is  currently  gaining  attention 
recommends  seeking  out  companies  with  generic  systems  integration  expertise,  as 
signalled  by  the  UK  selection  of  W.S.  Atkins  for  the  British  Future  Rapid  Effects  System 
(FRES)  programme.  That  said,  it  has  been  observed  that  the  Lead  Systems  Integrator  role 
in  the  US  has  driven  companies  to  seek  to  win  contracts  only  in  weapons  areas  in  which 
they  have  little  specialist  expertise  (Moon  &  Schoder,  2005). 

Perhaps  the  optimum  approach  is  a  more  modest  one,  in  which  separate  systems 
are  procured,  then  integrated  through  a  series  of  ad  hoc  patches;  these  may  be  in  the  form 
of  software  and  hardware,  or  they  may  involve  putting  a  human  in  the  loop.  Indeed,  such  an 
approach  aligns  well  with  the  concept  of  Incremental  Acquisition  in  which  the  required 


23  Such  complexity  arises  not  only  within  an  environment  (e.g.,  Land,  Sea  or  Air),  but  because 
current  and  future  operations  increasingly  (indeed,  probably,  inevitably)  require  the  joint  effects  of  two 
(if  not  all  three)  Services. 

24  For  a  series  of  short  articles  dealing  with  the  acquisition  of  Network  Enabled  Capability,  see 
(2004,  July).  RUSI  Defence  Systems,  7(1),  pp.  62-85. 
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capability  is  achieved  through  a  number  of  relatively  small  (and,  thus,  from  a  scientific  or 
engineering  perspective,  less  risky)  advances.  This  has  the  benefit  of  allowing  adjustments 
in  emphasis  to  be  made  as  the  nature  of  the  threat  unfolds,  but  invariably  involves  additional 
expense  not  provided  for  in  the  budgets  of  individual  projects. 

The  centralisation-delegation  pendulum  has  undoubtedly  swung  back  and  forth  over 
the  last  five  years  with  the  initial  development  of  a  voluntary  “Support  Solutions  Envelope” 
designed  to  curtail  the  more  extreme  approaches.  However,  this  is  now  being  modified,  and 
the  current  trend  is  clearly  towards  constraining  the  IPT  Leaders  rather  than  giving  them 
greater  discretion,  but  this  of  course  means  that  focus  on  and  commitment  to  achieving  the 
specific  project  goals  may  be  weakened.  Striking  the  right  balance  between  empowerment 
of  the  IPT  Leader  to  develop  novel  solutions  and  achieving  coherence  of  support  for,  say,  all 
the  equipment  in  a  Brigade  Group  is  a  fascinating,  but  extremely  challenging,  balancing  act. 
Furthermore,  as  indicated  earlier,  the  balance  chosen  is  likely  to  reflect  cultural  and  other 
environmental  factors;  for  example,  from  an  external  perspective,  the  US  defence  machine, 
in  both  operational  and  managerial  sectors,  often  appears  more  centrally  directed  and  rule- 
bound  than  that  found  in  the  UK. 

Another  area  in  which  the  Empowerment  v  Coherence  tensions  can  be  readily 
observed  is  in  the  subtle  balance  between  the  role  of  the  single  Service  Commanders-in- 
Chief  and  that  of  the  Chief  of  Joint  Operations.  Current  UK  operational  thinking,  supported 
by  recent  experience  in  several  theatres,  emphasises  the  need  for  “jointery” — but,  for  good 
reason,  the  peacetime  programme  designed  to  ensure  the  maintenance  of  a  particular 
capability  tends  to  be  undertaken  on  a  component  (i.e.,  Navy/Army/Air  Force)  basis.  This 
leads,  not  least,  to  the  reinforcement  of  cultural  differences  that  need  to  be  quickly 
overcome  when  a  joint  force  is  fielded.  The  UK  is  becoming  increasingly  aware  of  the  need 
to  approach  both  managerial  and  operational  issues  from  a  joint  perspective,  but  this 
prescription  may  not  be  appropriate  to  every  country.  Nevertheless,  many,  if  not  all,  will 
need  to  develop  the  appropriate  mechanism  to  ensure  the  coherence  of  military  output 
without  diminishing  the  essential  differences  between  the  components. 

Challenge  2 — What  should  the  governmental  defence  sector  do 
for  itself,  and  what  should  it  outsource  from  others? 

Even  in  the  US,  in  modern  times,  western  governments  do  not  develop  and  produce 
all  the  goods  and  services  that  their  armed  forces  need.  This  is  equally  true  of  states  such 
as  France,  Italy  and  Turkey,  where  notwithstanding  the  presence  (even  in  the  early  21st 
Century)  of  a  large  nationalised  defence  industrial  sector,  their  Ministries  of  Defence  still 
look  to  private  firms  for  the  provision  of  the  sort  of  products  used  by  the  general  population 
(such  as  food),  as  well  as  those  emanating  from  high-technology  sectors  (such  as 
electronics  and  aerospace).  However,  driven  mainly  by  a  belief  that  private  firms  working  in 
a  competitive  environment  are  more  efficient  than  publicly  owned  monopolies,  some 
countries  (with  the  UK  in  the  lead)  have  undertaken  a  significant  programme  of  privatisation 
of  their  state-owned,  defence-related  industries.  For  example,  over  the  last  two  decades, 
the  Royal  Dockyards  and  Royal  Ordnance  Factories  have  become  fully  fledged  businesses 
within  the  private  sector;  indeed,  even  the  UK’s  nuclear  weapons  plants,  whilst  still  formally 
owned  by  the  government,  are  managed  and  operated  for  the  government  by  a  private 
contractor. 
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Furthermore,  the  UK  (along  with  the  US  and  others)  has  increasingly  outsourced  the 
design,  development  and  production  of  defence  equipment  to  the  private  sector.  In  addition, 
the  UK  Ministry  of  Defence  has  not  been  immune  from  the  general  pan-Whitehall  drive  to 
implement  the  government’s  Private  Finance  Initiative  (PFI)  and  Public-private 
Partnerships25  (PPP).  For  example,  a  recent  review  (RUSI,  2004)  indicated  that  between 
1995  and  December  2003,  the  MOD  had  signed  contracts  for  45  PFI  projects  and  6  PPP 
arrangements — involving  some  £3.5  billion  of  capital  costs,  and  with  the  annual  payments 
representing  some  6%  of  the  Defence  budget.  Furthermore,  as  an  indication  of  the 
increasing  momentum  behind  this  approach,  another  37  projects  (worth  some  £12  billion) 
were  at  that  time  either  under  consideration  or  in  the  process  of  going  to  contract. 

Unsurprisingly,  the  areas  of  business  initially  transferred  out  were  those  providing 
support  services  such  as  cleaning  and  catering.  This  was  followed  by  a  second  wave  of 
projects  covering  a  broad  swathe  of  training  functions,  including,  for  example,  that  for 
armoured  vehicle  drivers  and  for  helicopter  and  fixed-wing  aircraft  pilots.  However,  recent 
initiatives  have  been  significantly  more  ambitious  and  are  increasingly  linked  with  the  overall 
model  for  the  transition  of  defence  support. 

In  essence,  in  developing  this  policy,  the  then-UK  Chief  of  Defence  Logistics  (CDL) 
believed  that  in  order  to  improve  the  reliability  of  military  equipment — with  the  concomitant 
beneficial  effects  in  terms  of,  for  example,  a  reduction  in  both  Through-life  Costs  and  a 
reduced  logistics  footprint  on  operations — it  was  necessary  to  engage  commerce  and 
industry  more  closely  in  the  delivery  of  military  effect.  The  net  result  has  been  a  developing 
generation  of  PFIs  that  sees  the  capital  cost  of  military  equipment  being  borne  by  industry — 
which  also  provides  ongoing  support  in  the  shape  of  some  or  all  of  maintenance,  training  (of 
both  operators  and  military  engineering  staff),  management  of  obsolescence  and  provision 
of  spare  parts.  Recent  examples  of  this  include  the  Skynet  5  satellite  system,  the  £600 
million  contract  for  the  provision  of  so-called  “C”  vehicles26  announced  in  June  2005,  and  the 
Future  Strategic  Tanker  Aircraft  (FSTA)  which  requires  industry  to  raise  some  £3  billion — 
making  this,  by  far,  the  largest  UK  PFI  ever  contemplated. 

Significantly,  the  use  of  contractors  for  many  of  these  roles  is  not  restricted  to  home 
activities,  but  also  extends  to  deployed  operations.  As  a  result,  there  is  now  a  real  debate 
as  to  the  extent  to  which  governments  should  rely  on  Contractors  on  Deployed  Operations 
(CONDO)27.  For  example,  Ukrainian  companies  have  become  major  suppliers  of  air 


26  Whilst  there  are  financial  and  accounting  distinctions  between  PFI  and  PPP  programmes,  the 
essence  of  both  is  that  private  (i.e.,  non-government)  funding  is  used  to  provide  the  capital  cost  of, 
say,  a  new  building.  This  is  paid  back  by  defence  over  the  long  term  (typically  in  excess  of  20  years) 
by  means  of  a  stream  of  rental  income.  In  all  probability,  the  totality  of  these  annual  payments  will 
exceed  the  cost  of  in-house  provision,  but  this  is  tolerated  not  only  because  of  the  extent  to  which 
“spikes”  in  capital  expenditure  (which  represents  some  25%  of  the  Defence  budget)  can  be  reduced, 
but  also  because  of  the  beneficial  effects  on  the  Public  Sector  Borrowing  Requirement  (PSBR)  and 
the  reduced  headcount  that  is  directly  funded  from  the  government’s  payroll. 

26  Military  vehicles  can  be  broken  down  into  generic  groups:  “A”  vehicles  include  armoured 
platforms  such  as  tanks.  “B”  vehicles  cover  the  soft-skin  group  such  as  4x4  vehicles.  The  “C”  vehicle 
category  includes  engineering  plant  such  as  bulldozers  and  earthmovers.  The  final  category  is 
known  as  the  “White  Fleet,”  which  covers  all  forms  of  staff  cars  and  minibuses  which  are  already 
provided  through  a  PFI  contract. 

27  See,  for  instance,  a  series  of  articles  in  (2004,  Summer).  Defence  Studies,  4(2). 
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transport  to  the  UK  defence  establishment,  while  Kellogg,  Brown  &  Route  holds  an  enabling 
contract  (known  as  “CONLOG”)  to  provide  a  whole  range  of  services  in  support  of  UK 
operations,  and  the  next  generation  of  air-to-air  refuelling  aircraft  is  highly  likely  to 
contracted  under  a  CONDO  arrangement.  The  leading  edge  of  this  generation  of  CLS 
initiatives  is  that  of  the  Heavy  Equipment  (e.g.,  Tank)  Transporter  (HET)  fleet,  in  which  1/3  of 
the  drivers  and  maintainers  are  civilians  who,  as  a  condition  of  their  employment,  must 
accept  Reservist  status.  The  benefit  of  this  approach  is  that  the  drivers  can,  at  short  notice, 
be  “re-badged”  as  military  personnel  with  minimal  interruption  to  the  level  of  support 
provided.  A  similar  approach  has  recently  been  agreed  for  the  RAPIER  Ground-to-Air 
missile  system;  the  operationally  deployable  2nd  Line  test  and  repair  facility  formerly  manned 
by  military  technicians  is  now  being  provided  by  the  company  (MBDA)  using  the  “sponsored 
reserve”  concept. 

In  pursuing  the  outsourcing  route,  the  UK  MOD  is  clearly  following  guidance  from  the 
literature  of  commercial  management.  For  example,  writers  in  this  field  such  as  Hamel  & 
Prahalad  (1994)  and  Johnson,  Scholes  and  Whittington  (2005)  recommend  that  companies 
should  seek  to  identify  their  “core  competencies” — i.e.,  areas  in  which  they  excel  and,  thus, 
hold  competitive  advantage.  These  authors  assert  that  companies  should  contract  with 
others  that  have  particular  expertise  for  other  necessary,  but  non-core,  functions. 
Unfortunately,  what  is  lacking  currently  in  the  UK  (and,  arguably  in  most  Western  nations)  is 
any  clear  view  about  what  the  core  competencies  of  the  governmental  defence  sector  are 
and  need  to  be. 

A  further  implication  of  outsourcing  is  that  it  changes  the  nature  of  one  of  the  more 
significant  challenges  facing  any  organisation — namely  how  to  select  one’s  suppliers.  There 
are  many  prescriptions  developed  by  academics  and  practitioners,  but  no  sure  route  to 
success.  Outsourcing  should  not  be  used  to  avoid  a  difficult  problem,  and  defence,  which 
buys  complex  products  and  services,  is  certainly  taking  on  risk  if  it  entrusts  provision  to 
prime  contractors  without  detailed  oversight  of  how  those  primes  will  deal  with  the  supply 
chains  below  them.  In  many  cases,  the  UK  has  little  choice  regarding  a  prime,  since  BAES 
is  the  supplier  of  90%  of  land  platforms  and  80%  of  fixed-wing  aircraft,  and  so  cannot 
introduce  supply  chain  management  as  a  significant  element  in  a  competitive  tendering 
activity. 

One  area  in  which  the  MoD  (but  not  the  Government  as  a  whole)  has  drawn  the  line 
is  in  its  refusal  to  place  lethal  force  in  the  hands  of  contractors,  though  contractors  are 
certainly  being  placed  nearer  to  the  front  line.28  However,  it  has  been  reported  that  the 
Royal  Norwegian  Navy’s  Fridtjoff  Nansen  class  frigates  are  taking  outsourcing  to  novel 
territory.  They  will  remain  in  the  ownership  of  IZAR,  their  Spanish  shipbuilders,  who  will  be 
not  only  responsible  for  the  support  and  maintenance  of  the  ships,  but  will  also  provide  one 
third  of  the  crew  (Cushway,  2006). 

This  increasing  use  of  CONDO  underlines  two  challenges.  First,  there  is  a  need  to 
ensure  the  integration  of  the  output  of  potentially  disparate  groups  of  contractors  to  provide 


28  Interestingly,  the  US  Department  of  Defense  (DoD)  has  gone  even  further  by  accepting 
that  private  military  companies  can  arm  their  personnel  for  such  tasks  as  guarding  oil  fields 
and  associated  facilities  such  as  pumping  installations,  etc.  Indeed,  there  may  be  as  many 
as  20,000  armed  guards  in  Iraq  under  US  contracts — albeit  many  are  not  of  American  or 
Iraqi  nationality. 
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“joined  up”  logistics  in  support  of  the  front-line  troops.  This  requires  greater  imposition  of 
common  standards  in  many  areas — including  information  systems,  health  and  safety,  and 
welfare/discipline,  etc.  Secondly,  and  perhaps  even  more  importantly,  it  raises  the  question 
of  how  the  services  that  they  are  delivering  will  be  provided  if  a  previously  benign  area 
becomes  markedly  more  dangerous.  This  is  a  difficult  balancing  act  in  this  era  of 
asymmetric  warfare —  particularly  if  the  UK  follows  the  US  lead  and,  for  example,  deploys 
contractors  in  direct  support  of  armoured  vehicles.  This  approach  is  used  to  support  its 
Stryker  Brigade,  in  which 

“Approximately  120  specialized  contractors  are  an  integral  part  of  the  Stryker 
Brigade  Combat  Teams’  (SCBTs)  highly  complex  systems  maintenance, 
sustainment  and  technical  support.  [...]  many  contractors  are  actually  operating  in 
the  forward  areas  of  the  SBCT”  (Alderete,  2005;  GAO,  2003,  June). 

A  decision  to  outsource  requires  not  just  the  confidence  that  an  external  supplier 
should  be  able  to  provide  a  capability  or  service  more  effectively  and  efficiently.  There  must 
be  confidence  that  the  contractor  can  be  incentivised  to  perform  reliably,  even  when  the 
physical  and/or  political  environment  has  become  challenging.  This  is  obviously  most 
relevant  in  the  military  operational  context,  but  there  must  also  be  a  viable  procurement 
strategy  available  that  will  give  the  outsourcing  authority  confidence  that  value  can  be 
obtained.  This  is  increasingly  leading  to  the  development  of  “Partnering”  solutions  that 
combine  elements  of  both  PPP/PFI  and  Outsourcing. 

Challenge  3 — How  should  support  for  equipment  be  arranged  in 

a  time  of  frequent  and  surprising  operations? 

Across  the  broad  swathe  of  UK  military  commentators,  there  is  a  clear  recognition 
that  the  current  defence  supply  chain  model  needs  to  be  significantly  developed  from  that 
originally  created  to  support  the  British  Army  when  facing  a  potential  Soviet  threat  on  the 
plains  of  Northern  Germany.  In  considering  how  it  should  be  improved,  planners  are  faced 
with  exhortations  that  supply  chains  should  be  “lean”  and  use  a  “just-in-time”  approach 
modelled  on  commercial  operations  such  as  those  providing  fast-moving  consumer  goods 
(FCMG).  Such  a  prescription  is  potentially  attractive,  not  least  as  it  is  believed  that  it  will 
enable  a  reduction  in  the  existing  stockpiles  and,  hence,  reduce  financial  overheads. 
However,  the  Armed  Forces,  whose  lives  depend  on  stocks  of  ammunition  and  spare  parts 
being  replenished  promptly,  have  traditionally  preferred  a  “just-in-case”  approach.  This 
feeling  is  well  captured  by  the  then-US  Assistant  Secretary  for  Defense  who  observed  that, 
“In  the  absence  of  rock  solid  information  regarding  the  availability  of  materiel,  the  warfighter 
will  always  buy  readiness  insurance  in  the  form  of  excess  local  stocks”  (Kaminski,  1996). 

On  the  other  hand,  carelessness  or  mismanagement  can  also  lead  to  stocks  being  held  for 
contingencies  that  have  long  become  unthinkable.  For  instance,  the  establishment  of  the 
UK’s  Defence  Logistics  Organisation  (DLO)  in  2000  led  to  the  discovery  of  jigs  and  tools  that 
would  have  supported  the  re-launched  production  of  2nd  World  War-type  aircraft! 

From  an  academic  perspective,  the  reluctance  to  embrace  the  “just-in-time”  (JIT) 
model  is  entirely  logical,  as  this  concept  operates  best  when  demand  is  relatively  stable  and, 
hence,  predictable  (Towill  &  Christopher,  2002).  This,  unsurprisingly,  sits  uncomfortably 
with  the  doubly  unpredictable  nature  of  warfare,  in  which  we  cannot  be  confident  about  the 
timing  and  location  of  military  operations,  or  about  precisely  how  they  will  unfold  once  they 
have  begun.  Thus,  the  alternative  “Agile”  supply  chain  management  model  would  appear 
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more  promising  in  a  military  context  as  it  recognises  that  all  forecasts  are  inherently 
imperfect  and  is,  therefore,  designed  “to  thrive  and  prosper  in  an  environment  of  constant 
and  unpredictable  change”  (Maskell,  2001). 

The  conflicts  in  Iraq  and  Afghanistan  are  exceptional  and,  since  1945,  the 
requirement  for  the  UK’s  Armed  Forces  to  prosecute  the  Queen’s  enemies  has  been  applied 
infrequently.  Certainly,  large  numbers  of  UK  troops  are  often  engaged  in  hazardous 
missions,  but  the  periods  of  time  in  which  they  are  fighting  in  a  major  conflict  (as  distinct 
from  Peace  Support  or  Peace  Enforcement  Operations)  are  relatively  short — perhaps  for 
some  six  months  during  a  ten-year  window.  Therefore,  use  of  an  efficient  business  model 
makes  good  sense  in  peacetime  when  the  armed  forces  are,  typically,  engaged  in  routine 
training  in  order  to  ensure  that  they  develop  and  maintain  their  expertise.  The 
consequences  of  a  vehicle  breakdown,  and  any  delay  to  its  repair  or  replacement,  may 
result  in  a  waste  of  valuable  resources  and  be  very  frustrating  for  all  concerned — but  rarely 
is  life  put  at  risk.  Contrast  this  with  the  operational  situation  in  which  supply  chain  failures 
can,  and  regrettably  do,  lead  directly  or  indirectly  to  death  or  injury.  Thus,  the  operational 
supply  chain  must  be  optimised  towards  effectiveness  (with  its  certainty  of  supply)  rather 
than  efficiency. 

Thus,  the  Defence  Supply  Chain  sits  firmly  on  the  horns  of  a  dilemma:  whether  to 
reduce  inventories  in  order  to  reap  the  peacetime  efficiency  benefits,  or  to  continue  to  pay 
this  “insurance”  cost  in  order  to  help  ensure  the  effectiveness  essential  for  successful 
operations.  In  theory,  there  will  be  an  optimum  level  of  “leanness”  (Christopher  & 

Rutherford,  2004) — but  ascertaining  this  for  each  of  1 .7  million  SKUs,29  in  the  face  of  the 
uncertain  future  demand  pattern  that  is  the  inevitable  consequence  of  the  uncertain  nature 
of  future  warfare,  may  well  be  beyond  even  a  significant  investment  in  sophisticated 
modelling.  Hence,  it  is  unsurprising  that  the  military  response  to  a  combination  of  uncertain 
demand  and  long-lead  time  supply  characteristics  is  likely  to  continue  to  result  in  significant 
stockpiling — but  it  is  just  such  stockpiling  that  then  becomes  a  target  for  challenge  in 
subsequent  spending  rounds. 

To  the  stresses  between  JIT  and  Agile  must  be  added  the  dimension  of  resilience: 
that  supply  chains  should  be  able  to  avoid  or  absorb  shock.  Arguably,  defence  needs  to 
take  evaluate  risks  to  effective  supply,  including  reliance  on  single  plants  that  may  be 
destroyed  in  fires  or  other  accidents.  Stocks  in  different  locations  can  clearly  enhance 
resilience,  but  such  strength  rarely  comes  without  cost.  Costs  must  always  be  weighed 
against  the  perceived  risks — but  persuading  those  who  control  the  defence  purse  strings  of 
the  merits  of  these  observations  generally  remains  an  unmastered  challenge. 

Overall,  however,  the  response  of  the  UK  MOD  has  been  to  enhance  the  importance 
of  potential  operations  in  the  planning  and  management  of  the  DE&S  organisation.  The  UK 
also  is  giving  increased  recognition  to  security  of  supply  in  times  of  crisis — for  instance  in  the 
1998  Letter  of  Intent  with  European  States30  and  the  2000  Declaration  of  Principles  with  the 
United  States.  Perhaps  significantly,  Switzerland — a  relatively  friendly  country — placed  an 


29  SKUs  =  Stock  Keeping  Units.  The  UK’s  defence  inventory  should  be  compared  with  the  20- 
30,000  SKUs  that  might  be  found  in  a  typical  supermarket  outlet  (Fernie  &  Sparks,  1999). 

30  It  is  planned  that  this  Letter  of  Intent  will  be  developed  through  the  introduction  of  EU  Code  of 
Conduct  on  Defence  Procurement  (MOD,  2005,  p.  7). 
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embargo  on  the  export  of  grenades  to  the  UK  as  a  reflection  of  its  government’s  disapproval 
of  the  2003  Iraq  War  (NAO,  2003).  Fortunately,  this  had  no  tangible  effect  on  the  outcome 
as  sufficient  had  been  stockpiled,  but  it  was  a  salutary  lesson  for  planners. 

It  is  suggested  that  the  keys  to  the  MOD’S  difficulty  here  are  threefold.  Firstly,  there 
has  been  a  solid  record  of  recent  operational  success.  It  is  argued,  therefore,  that 
“warts  and  all,”  the  current  system  which,  whilst  clearly  not  perfect,  may  be  just  about 
as  good  as  it  will  get  given  the  variables  in  play.  This  “do  nothing”  (other  than  to 
continue  to  tune  the  system)  approach  clearly  has  advantages  for  those  within  the 
MOD  and  industry  whose  interests  are  well  served  by  it.  Challenging  these  power 
bases  without  a  clear  idea  of  what  the  successor  system  will  look  like  is  a  career- 
threatening  move  in  any  arena! 

Secondly,  the  lead  time  for  a  significant  volume  of  materiel  support  is  long  and,  in 
many  cases,  growing.  This  is  generally  the  result  of  the  niche-status  of  defence  engineering 
(both  mechanical  and  electronic)  and  its  relative  lack  of  players  (as  discussed  above). 

Given  that  defence  is  generally  recognised  as  not  following  the  normal  economists’  model  of 
a  perfect  market,  there  is  only  limited  pressure  that  can  be  placed  on  those  remaining 
companies  to  improve  their  performance — especially  in  the  time  dimension.  This 
exacerbates  the  pressure  on  individual  IPTs  to  take  the  easy  (and  generally  risk-free)  option 
of  developing  stockpiles  rather  than  seeking  more  agile  solutions.  But  as  the  move  towards 
partnering  takes  greater  hold  with  its  metrics  based  around  delivering  an  agreed-upon  level 
of  equipment  availability,  this  decision  is  transferred  to  the  industrial  partner  that  must 
increasingly  make  such  stock-level  decisions.  Experience  to  date  would  indicate  that  such 
partners  are  up  to  this  task,  but  there  is  a  large  and  diverse  body  of  academic  literature  that 
raises  fundamental  concerns  over  the  effectiveness  of  such  collaboration  arrangements 
(Kampstra,  Ashayeri  &  Gattorna,  2006).  Thus,  the  inevitable  nervousness  of  operational 
commanders  over  this  fundamental  transference  of  risk  may  yet  prove  well-placed. 

The  final  point  is  that  the  capability-based  approach  is  rare,  perhaps  even  unique  to 
Defence.  No  other  organisation  spends  almost  the  totality  of  its  budget  in  the  development 
of  a  capability  (across  equipment,  communications,  personnel,  training,  infrastructure, 
operational  planning,  etc)  and  then  returns  to  its  shareholders  (in  the  MOD’S  case  the 
Treasury)  for  additional  finance  when  they  are  actually  required  to  employ  this  capability. 
Even  major  public-sector  ventures  such  as  hospitals  and  schools  aim  to  balance  the  supply 
(of  operating  theatres,  nurses,  classrooms,  teachers)  against  steady  state  demand.  Peaks 
can  be  accommodated  through  a  number  of  mechanisms  (geographic  dispersion  of  the 
requirement,  building  temporary  facilities,  hiring  agency  staff)  in  relatively  short  order.  Thus, 
unlike  the  MOD  with  its  massive  equipment  lead  times,  the  risk  from  adopting  a  surge 
(rather  than  a  stockpile)  model  is  relatively  containable. 

As  a  result,  the  MOD  has  no  obvious  commercial  comparators  against  which  it  can 
benchmark  its  activities  and  is  generally  constrained  by  the  extremely  coarse  level  of  public 
expenditure.  That  said,  there  is  an  implicit  recognition  that  a  country  cannot  undertake 
major  military  operations  without  the  support  of  a  capable  industry.  But,  generally,  there  is  a 
reluctance  to  confront  too  squarely  the  additional  costs  that  have  to  be  incurred  in 
peacetime  to  ensure  that  sustainability  can  be  assured  in  war. 
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Challenge  4 — In  an  era  of  rapid  progress  in  civil  technology, 
how  does  a  Ministry  of  Defence  assure  its  optimum  inclusion 
into  defence  systems? 

Forty  years  ago,  research  and  development  in  the  West  was  dominated  by  military 
spending.  Today,  this  is  no  longer  the  case;  technological  advance  in  many  sectors, 
including  computing  and  IT  and  biotechnology,  is  almost  entirely  driven  by  civil  spending. 
Even  in  the  UK,  where  defence  Research  and  Development  (R&D)  is  comparatively 
important  compared  with  other  EU  members  (except  France),  defence  accounts  for  only 
1 1  %  of  the  national  R&D  effort  (National  Statistics,  2005).  This  increasing  focus  on  the  R&D 
requirements  of  commerce  and  industry  raises  the  fundamental  question  of  how  defence 
can  minimise  the  chances  that  it  will  not  miss  out  on  opportunities  that  may  arise  in  the 
civilian  context. 

As  part  of  its  response  to  this  challenge,  the  UK  MOD  devotes  some  time  and 
resources  simply  to  tracking  progress  in  civil  technology  across  the  board  and  across  the 
world.  It  also  operates  a  small  organisation  within  the  DE&S  (the  Defence  Suppliers’ 
Service)  that  familiarises  businesses  seeking  work  in  the  defence  sector  with  the  contractual 
and  other  processes  of  the  Ministry  of  Defence  and,  in  this  way,  tries  to  ease  the  path  of 
firms  that  wish  to  become  suppliers.  On  the  other  hand,  the  US  Department  of  Defense  has 
particular  accounting  demands  that  it  places  on  its  suppliers;  this  has  led  to  concerns  that 
some  generators  of  “civil”  technology  do  not  promote  it  in  the  defence  sector  simply  because 
they  do  not  want  the  trouble  of  complying  with  these  accounting  systems. 

Little  in  this  area  is  easy.  Much  contemporary  technology — especially  that  which 
relies  heavily  on  electronics — has  a  very  short  lifecycle  and  is  kept  in  production  for  only  a 
limited  time.  Culturally  and  procedurally,  the  UK  MOD  has  developed  an  increasingly  robust 
process  for  the  careful  assessment  of  equipment  investment  that  is  able  to  handle 
programmes  with  a  planned  lifetime  of  two  or  three  decades  (albeit  many,  with  subsequent 
life  extension  programmes,  have  the  period  from  Concept  to  Disposal  often  exceeding  50 
years  (DEG,  2005)).  The  Investment  Approvals  Board31  (IAB)  is,  therefore,  not  accustomed 
to  hearing  that  a  piece  of  kit  will  be  disposed  of  after  perhaps  three  or  four  years — which  is 
frequently  shorter  than  the  approvals  process  itself! 

Furthermore,  the  building  of  modern  computing  and  information  technology  into 
complex  larger  systems  also  raises  questions  about  whether  and  how  that  technology  can 
be  updated  without  significant  disruption  to  the  system  as  a  whole.  For  example,  modern 
combat  aircraft  such  as  the  F22  and  the  Eurofighter  Typhoon  contain  several  thousand 
obsolescent  parts,  but  changing  a  486  processor  for  a  modern  Pentium  version  is  a  risky 
business  in  such  complex  systems.  To  date,  the  UK  Defence  Ministry’s  answer  to  this  issue 
has  been  twofold.  In  some  cases,  a  lifetime  supply  of  an  item  is  bought  at  the  time  of 
procurement  so  that  a  failed  component  can  be  simply  replaced  from  existing  stock. 
However,  this  has  the  obvious  downsides  of  the  both  the  cost  of  maintaining  such  a 
stockpile  (particularly  when  its  usage  rates  may  prove  to  be  far  lower  than  estimated  at  the 
pre-production  stage),  and  also  its  vulnerability  to  the  sort  of  obsolescence  issues  outline 
above. 


31  As  its  name  implies,  the  IAB  is  the  committee  that  sanctions  proposals  for  major  investment  on 
behalf  of  the  UK’s  Defence  Management  Board. 
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The  alternative  (and  increasingly  common)  approach  is  to  develop  a  support 
arrangement  with  a  contractor  who  is  obliged  both  to  maintain  the  system  and  to  update  its 
elements  under  a  so-called  “Contracting  for  Availability”  approach.  This  is  particularly  useful 
in  the  procurement  of  information  systems  and  services  and,  as  a  result,  the  contractor 
shoulders  the  risk  of  difficulties  associated  with  the  introduction  of  new  technology.  Given 
that  the  contractor  is  usually  the  original  equipment  supplier,  he  should  (in  theory) 
understand  the  system  better  than  anyone  else.  But,  with  the  increasing  importance 
accorded  to  the  electronic  dimension  of  a  platform  such  as  a  tank,  ship  or  aircraft,  it  is  not 
surprising  that  that  some  commentators  have  gone  so  far  as  to  suggest  that  the  electronics 
should  lead  the  design:  “In  future,  we  will  invite  contractors  to  design  C4ISTAR  architectures 
and  integrate  platforms  into  them”  (Blackham,  200332). 

Challenge  5 — Development  of  a  Whole-life  and  pan- 
Organisation  Cost  mentality 

In  an  overall  “system  of  systems”  as  complex  as  Defence  (which  is  designed  to 
operate  successfully  at  some  distance  from  its  home  base  in  the  most  challenging  and  life- 
threatening  of  environments),  it  is  inevitable  that  there  will  be  overlaps  and  interfaces 
between  those  charged  with  the  management,  development  and  operation  of  each 
individual  component.  Thus,  for  example,  the  introduction  of  an  improved  equipment  design 
is  likely,  whilst  reducing  the  direct  cost  (of  operation  and/or  spares  provision),  to  incur 
additional  costs  in  terms  of  infrastructure  (e.g.,  more  sophisticated  diagnostic  equipment) 
and  in  personnel  (training  of  maintenance  engineers  to  higher  skill  levels). 

Indeed,  in  the  UK,  the  military  has  recognized  eight  so-called  Defence  Lines  of 
Development33  (LoDs)  as  a  means  of  helping  ensure  that  the  implications  of  equipment 
changes  are  well  understood.  The  simplest  approach  sees  equipment,  at  least  after  the 
signature  of  a  development  contract  with  a  company,  as  the  independent  variable  from 
which  other  LoDs  follow:  their  management  requires  their  timely  and  appropriate  provision. 

A  more  complex  line  of  thought  presents  the  LoDs  as  interacting  throughout  the 
development  process;  for  instance,  a  computer  programme  could  be  developed  expensively 
with  few  training  needs  or  more  cheaply  with  extensive  training  needs.  However,  viewing 
the  LoDs  in  this  way  presents  significant  challenges  in  the  development  of  robust  and 
meaningful  contracts  with  industry.  Intriguingly,  as  recognised  by  the  UK’s  Chartered 
Institute  for  Logistics  and  Transport,  the  LoD  approach  does  seem  to  commend  itself  as  a 
model  in  other  areas — for  example,  in  planning  responses  to  Humanitarian  Disasters  in 
which  the  integration  of  many  strands  of  support  (against  the  backdrop  of  uncertainty, 
potentially  devastated  infrastructure  and  limited  communications)  has  clear  parallels  with  the 
military  scenario. 


32  Vice  Admiral  Sir  Jeremy  Blackham  was,  until  2003,  the  officer  responsible  for  the  development  of 
future  equipment  capability — the  so-called  “Customer  1,”  whose  role  included  the  development  of  the 
User  Requirements  Document  (URD)  against  which  new  platforms  and  equipment  are  procured. 

33  Known  by  the  acronym  “TEPIDOIL,”  these  encompass  Training,  Equipment,  Personnel, 
Infrastructure,  Doctrine  &  Concepts,  Organisation,  Information  and  Logistics. 
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A  further  twist  to  this  multi-faceted  integration  problem  is  the  increasing  recognition 
that  resource-expenditure  decisions  should  be  considered  on  a  Whole-life  cost34  (WLC) 
basis.  In  many  ways,  it  would  seem  to  be  entirely  perverse  not  to  approach  such  decisions 
from  this  perspective;  however,  there  are  many  examples  of  procurement  projects  in 
defence  in  which  the  capital  cost  would  appear  to  have  been  deliberately  reduced  without 
formally  recognising  the  implications  for  the  ongoing  costs  of  support,  and  in  the  hope  that 
the  latter  will  be  “lost  in  the  noise”  or  that  the  corporate  memory  will  have  forgotten  the 
original  basis  for  the  decision  when  it  crystallises  20  years  later. 

It  is  fully  accepted  that  Whole-life  costing  is  an  acknowledged  challenge  in  many 
areas  of  commerce  and  industry,  but  the  complexity  of  military  equipment  and  the  breadth  of 
operational  circumstances  in  which  it  can  be  (and  is)  used,  make  cost  forecasting  a 
particular  challenge.  As  an  example,  many  items  of  defence  equipment  are  designed  for 
use  in  one  environment  but,  due  to  changed  political  or  military  circumstances,  are  actually 
operated  in  a  markedly  different  way.35  This  complexity  is  exacerbated  by  the  extent  to 
which  organisational  structures  influence  the  potential  for  “tribal”  behaviour.  Thus,  in  the  UK 
MOD,  there  has  been  a  clear  separation  between  those  who  are  charged  with  the 
procurement  of  new  equipment,  those  who  maintain  it,  and  those  who  operate  it  in  combat. 
Particularly  in  relation  to  the  first  two  protagonists,  this  schism  has  reinforced  the  suspicion 
that — not  least  in  the  face  of  sustained  criticism  by  the  House  of  Commons  Public  Accounts 
and  Defence  Committees — misleading  trade-offs  between  capital  and  support  costs  may  be 
presented  to  scrutineers. 

The  merger  during  2007  of  the  Defence  Procurement  Agency  and  the  Defence 
Logistics  Organisation — as  well  as  wider  responsibilities  placed  on  the  Equipment  Capability 
Customer  organisation — is  meant  to  support  the  early  adoption  of  realistic  whole-life  cost 
calculations.  Robust  methods  for  capturing  and  allocating  such  costs,  however,  will  need 
further  refinement.  Nevertheless,  to  the  extent  that  the  need  for  Whole-life  Costing  across 
all  the  Defence  LoDs  has  been  recognised  and  will  lead  to  greater  attention,  the  Defence 
Industrial  Strategy  must  be  applauded. 

Conclusion 

This  paper  has  sought  to  review,  from  a  largely  UK  perspective,  some  of  the  key 
challenges  currently  facing  the  management  of  defence  and,  in  doing  so,  has  addressed  five 
inter-related  questions  to  which  there  are  no  easy  or  final  answers.  But,  this  study  has  also 
suggested  that  the  current  relative  paucity  of  academic  endeavour  presents  a  tremendous 
opportunity  both  for  new  insights,  and  also  for  the  development  and  application  of 
prescriptions  that  have  been  tried  and  tested  in  the  commercial  arena.  Furthermore,  whilst  it 


34  A  very  useful  and  concise  exploration  of  some  of  the  key  issues  surrounding  the  development  of 
robust  WLC  models  is  to  be  found  in  the  Kirkpatrick  (2000). 

35  A  particularly  clear  example  is  the  Royal  Navy’s  Type  23  Frigates  that  were  designed  for  anti¬ 
submarine  patrols  to  be  conducted  at  slow  (and,  therefore,  quiet)  speed  in  the  waters  between 
Greenland,  Iceland  and  Northern  Scotland.  Thus,  given  the  predicted  ambient  temperatures,  the  air 
conditioning  systems  (for  both  equipment  and  crew)  were  limited,  as  was  the  endurance — reflecting  a 
concept  of  operations  that  saw  the  vessels  being  accompanied  by  an  oiler/stores  support  ship. 
Following  the  end  of  the  Cold  War,  the  ships  are  now  seen  as  “General  Purpose”  frigates  and  have 
required  expensive  retrofitting  to  enable  them  to  operate  effectively  in,  say,  the  Persian  Gulf. 
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is  not  appropriate  to  speak  of  a  crisis  in  defence  management,  there  is  little  doubt  that  major 
shortcomings  in  existing  ways  of  doing  business  (competitive  tendering,  outsourcing,  and 
relying  on  rather  autonomous  Integrated  Project  Teams  in  acquisition)  are  becoming  ever 
clearer. 

Finally,  the  authors  are  well  aware  that  our  selection  does  not  cover  all  of  the 
management  challenges  facing  the  UK  MOD.  For  instance,  it  is  not  clear  whether  the 
benefits  gained  from  the  introduction  of  Resource  Accounting  and  Budgeting  (RAB)  have 
exceeded  the  considerable  implementation  costs  involved.  That  said,  there  is  no  doubt  that 
the  MOD  has  much  to  do  if  it  is  to  better  understand  its  own  costs  (on  a  Whole-life  basis  and 
across  all  aspects  such  as  personnel,  training,  support  and  infrastructure)  as  a  means  of 
improving  the  significant  balance  of  investment  decisions  that  it  faces.  Also,  the  prevalence 
and  successful  achievement  of  Urgent  Operational  Requirements  (UORs)  before  the  Iraq 
War  has  raised  questions  about  the  actual  outputs  and  outcomes  of  the  peacetime  defence 
budget,  as  well  as  the  procurement  process  as  a  whole — how  can  the  latter  meet  the 
demands  of  UORs  but  singularly  fail  to  do  so  in  regular  procurements  (especially  in  the 
absence  of  any  clear  price  inflation)? 

Hopefully,  the  issues  raised  in  some  detail  will  prompt  some  interest  in  the 
management  challenges  facing  the  defence  community.  They  might  even  generate  some 
sympathy  for  those  charged  with  running  a  country’s  defence  machine. 
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Abstract 

Several  previously  published  papers  have  cited  the  need  to  include  correlation  in 
risk-analysis  models.  In  particular,  a  landmark  paper  published  by  Philip  Lurie  and  Matthew 
Goldberg  presented  a  methodology  for  inducing  Pearson’s  correlation  between 
input/independent  random  variables.  The  one  subject,  absent  from  the  paper,  was  a 
methodology  for  finding  the  optimal  applied  correlation  matrix  given  a  desired  outcome 
correlation.  Since  the  publishing  of  the  Lurie-Goldberg  paper,  there  has  been  continuing 
discussion  on  its  implementation;  however,  there  has  not  been  any  presentation  of  an 
optimization  algorithm  that  does  not  involve  the  use  of  computing-heavy  simulations.  This 
paper  reviews  the  general  methodology  used  by  Lurie  and  Goldberg  (along  with  its 
predecessor  papers)  and  presents  a  non-simulation  approach  to  finding  the  optimal  input 
correlation  matrix,  given  a  set  of  marginal  distributions  and  a  desired  correlation  matrix. 

Introduction 

The  Complete  Correlation  Algorithm  (CCA)  developed  by  Northrop  Grumman  and 
recently  implemented  in  NG  developed  risk  models  is  a  product  of  more  than  two  years  of 
research  and  development.  Several  previously  published  papers  have  cited  the  need  to 
include  correlation  in  risk-analysis  models;  however,  none  present  an  optimization  algorithm 
that  does  not  involve  the  use  of  computing-heavy  simulations.  In  particular,  a  landmark 
paper  published  by  Philip  Lurie  and  Matthew  Goldberg  (1998)  presented  a  methodology  for 
inducing  Pearson’s  correlation  between  input  random  variables.  This  paper  reviews  the 
general  methodology  used  by  Lurie  and  Goldberg  (along  with  its  predecessor  papers)  and 
presents  the  Druker  Algorithm:  a  non-simulation  approach  to  finding  the  optimal  input 
correlation  matrix  given  a  set  of  marginal  distributions  and  a  desired  correlation  matrix. 

The  CCA  was  deliberately  created  bearing  in  mind  identified  environmental  factors 
that  prevent  easy  implementation  of  commercially  available  models.  No  one  on  the  team 
had  any  experience  implementing  correlation  into  Monte  Carlo  simulations  beyond  the  use 
of  COTS  programs,  such  as  @Risk™  and  Crystal  Ball™.  To  determine  the  best 
development  method,  the  following  factors  were  considered: 

1 .  The  Northrop  Grumman  risk  models  need  to  be  of  an  easily  transferable  electronic 
size,  as  the  models  are  often  shared  via  email  or  network  drives. 

2.  A  diverse  group  of  users  must  be  able  to  run  the  software  in  a  variety  of  work 
environments;  Microsoft  Office  is  the  only  platform  that  is  transferable  to  all  parties. 
Users  include  risk  practitioners,  program  managers  and  members  of  pricing 
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organizations;  locations  include  unclassified  and  classified  Northrop  Grumman 
facilities,  unclassified  and  classified  customer  facilities  and  home  offices. 

3.  Custom  implementations  are  frequent;  much  of  NGIT-TASC  risk  work  requires  risk 
simulations  to  be  built  into  pre-existing  cost  and  price  models.  These  models  are 
generally  limited  to  Microsoft  Excel  and  Access;  however,  Web-based  platforms  are 
not  unheard  of. 

The  above  concerns  drove  the  decision  to  use  Visual  Basic  source  code  to  develop 
the  CCA. 

Initially,  the  development  was  focused  on  an  algorithm  that  could  induce  Pearson’s 
correlation  between  typical  distributions  in  risk  analysis:  Bernoulli  (discrete),  Triangular, 
Normal  and  Log-Normal.  By  limiting  the  problem  to  the  most-common  applications,  in 
theory,  the  solution  should  have  been  easier  to  find.  While  attempting  to  ascertain  the 
maximum  correlation  between  any  two  Bernoulli  distributions,  however,  the  general  solution 
was  uncovered.  The  resulting  algorithm  induces  Pearson’s  correlation  between  any  set  of 
random  variables  (while  still  preserving  the  marginal  distributions)  using  the  Lurie-Goldberg 
Method  and  without  the  use  of  simulation  to  find  the  optimal  applied  correlation  matrix. 

The  CCA  is  a  compilation  of  multiple  algorithms  (each  named  for  their  main 
author(s))  from  several  sources:  existing  papers,  public  source  code  and  internally- 
developed  code.  Most  of  the  algorithms  used  were  taken  from  a  variety  of  existing  papers. 
Although  these  papers  all  provided  complete  algorithms,  they  sometimes  lacked  details  in 
how  to  accomplish  key  steps;  in  cases  such  as  these,  gaps  were  filled  with  open-source 
code  solutions.  The  optimization  of  the  applied  correlation  matrix,  the  last  step  in  the 
correlation  algorithm,  was  developed  entirely  by  the  Northrop  Grumman  Team. 

Definitions  and  Assumptions 

Matrix  Definitions: 

1 .  Consistent  Correlation  Matrix — Consistent  Correlation  matrices  have  diagonal 
entries  equal  to  1 .0,  all  other  entries  between  [-1 ,  1]  are  symmetric  and  positive 
definite.  Consistency  is  necessary  for  a  viable  correlation  matrix,  but  a  Consistent 
Correlation  Matrix  may  not  necessarily  be  viable  given  the  Parent  Distributions. 

2.  Input  Correlation  Matrix  (I) — The  user-inputted  correlation  matrix.  This  matrix  may 
or  may  not  be  a  consistent  correlation  matrix. 

3.  Adjusted  Correlation  Matrix  (L) — The  Input  Correlation  Matrix  adjusted  to  be  a 
Consistent  Correlation  Matrix.  This  matrix  will,  by  definition,  be  positive  definite. 
Additionally,  the  adjusted  matrix  will  be  viable  as  correlations  between  various 
distributions  of  random  variables  will  be  achievable.  When  (L)  is  generated,  the 
differences  between  (I)  and  (L)  are  minimized. 

4.  Applied  Correlation  Matrix  (A) — The  correlation  matrix  used  by  the  grand  algorithm 
to  generate  correlated  random  number  draws.  This  matrix  may  be  the  same,  or  very 
different  from,  the  Adjusted  Correlation  Matrix;  the  extent  of  the  differences  will 
depend  on  the  random  variables  to  be  correlated. 

5.  Optimal  Applied  Correlation  Matrix  (A’) — The  Applied  Correlation  Matrix  optimized 
using  the  Lurie-Goldberg  Method. 
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6.  Outcome  Correlation  Matrix  (O) — The  correlation  matrix  of  the  simulated  variables 
following  the  simulation  run.  The  goal  of  the  grant  correlation  algorithm  is  for  (O)  to 
be  identical  to  (L). 

Other  Definitions: 

1 .  Parent  Distribution — The  distributions  correlated  for  use  in  the  simulation.  The 
distributions  are  simulated  using  the  Inverse  CDF  technique.  The  goal  is  to  induce  a 
desired  correlation  between  these  distributions. 

2.  Pearson’s  Correlation — A  parametric  statistic  that  measures  the  strength  and 
direction  of  a  linear  relationship  between  two  random  variables  (“Correlation,”  2008). 

3.  Spearman’s  Rank  Correlation — A  non-parametric  statistic  that  measures  the 
monotonicity  of  a  function  without  making  any  assumptions  as  to  the  distribution  of 
the  variables. 

4.  Eigenvalues — A  scalar  (L)  associated  with  a  matrix  such  that  if  (A)  is  a  matrix  and 
(X)  is  a  vector,  AX  =  LX.  The  vector  (X)  is  known  as  the  Eigenvector  that 
corresponds  to  the  Eigenvalue  (L). 

Assumptions: 

1.  Normal  Distributions — Any  reference  to  the  normal  distribution,  whether  in  a 
univariate  or  bivariate  case,  is  assumed  to  be  the  Standard  Normal  distribution 
(Mean  of  0,  Standard  Deviation  of  1 ). 

Pearson’s  vs.  Rank  Correlation 

Most  COTS  risk  tools  use  Spearman’s  rank  correlation  as  a  substitute  for  Pearson’s 
correlation  between  parent  distributions.  Spearman’s  rank  correlation  (a  non-parametric 
statistic)  differs  from  Pearson’s  correlation  (a  parametric  statistic)  in  that  it  measures  the 
monotony  of  a  function,  whereas  Pearson’s  correlation  measures  the  strength  of  the  linear 
relationship  between  two  functions  (see  Figure  1).  Though  studies  have  shown  that,  using 
the  most  common  risk  distributions,  models  using  rank  correlation  yield  similar  results  to 
those  using  Pearson’s  (Robinson  &  Sails,  2004),  there  is  a  distinct  difference  between  the 
two.  Although  this  paper  will  not  detail  all  the  differences  between  the  two  measures,  a  quick 
(and  exaggerated)  example  is  presented  below.  The  grand  algorithm  supersedes  the  need 
to  substitute  for  Pearson’s  correlation  with  Spearman’s  rank  correlation. 
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Cost  vs.  Weight  y  -  69.857x  -  497.36 

R2  =  0.6554 


♦  Cost - Linear  (Cost) 


Pearson's  Rho 

0.81 

Spearman's  Rho 

1.00 

Figure  1.  Pearson's  vs.  Spearman's  Rank  Correlation 

Algorithm  Overview 

There  are  three  main  steps  behind  the  grand  algorithm.  An  outline  of  these  steps 
follows,  and  the  upcoming  sections  of  this  paper  will  review  each  individual  step  in  detail. 

1 .  Correct  the  User-Input  Correlation  Matrix  (I) 

a.  Correct  I  so  that  it  is  consistent — both  in  terms  of  a  general  correlation  matrix 
and  the  properties  of  the  parent  distributions  being  correlated. 

b.  Through  these  corrections,  the  Adjusted  Correlation  Matrix  (L)  will  be 
generated. 

2.  Optimize  the  Applied  Correlation  Matrix 

a.  Find  the  Optimal  Applied  Correlation  Matrix  (A’)  such  that  when  A’  is  run 
through  the  Lurie-Goldberg  Method,  the  Outcome  Correlation  Matrix  (O)  is 
identical  to  L. 

3.  Correlate  the  Input  Random  Variables 

a.  Using  A’,  apply  the  Lurie-Goldberg  Method  to  correlate  the  parent 
distributions. 

For  purposes  of  presenting  the  methodology,  it  is  necessary  to  show  how  the  input 
random  variables  are  to  be  correlated  before  discussing  how  to  find  A’. 
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Correcting  the  User-Input  Correlation  Matrix  (Part  I) 

Giving  users  the  ability  to  input  their  own  correlation  matrix  allows  for  the  possibility 
that  the  User-Input  Correlation  Matrix  (I)  may  not  be  a  viable  correlation  matrix. 

Correlation  matrices,  by  definition,  have  diagonal  entries  of  1.0.  All  other  entries  between  [- 
1,  1]  are  symmetric  and  are  positive  definite.  The  first  step  in  inducing  correlation  between 
input  random  variables  is  checking  whether  I  is  a  consistent  correlation  matrix.  If  it  is  not,  it 
must  be  corrected  that  it  is  such. 

The  Iman-Davenport  Algorithm,  which  is  based  on  a  paper  by  Ronald  Iman  and 
James  Davenport  (1982)  is  used  to  correct  I  in  order  to  make  it  a  consistent  correlation 
matrix.  While  numerous  other  papers  have  been  published  describing  methods  to  correct  I 
such  that  it  is  altered  as  little  as  possible  (Higham,  2002),  the  Iman-Davenport  Algorithm  is 
the  most  computationally  efficient  method  the  authors  uncovered.  Given  that  additional 
adjustment  may  be  required  based  on  the  parent  distributions  being  correlated;  the  resulting 
matrix  is  close  enough  to  I  to  satisfy  this  requirement. 

The  algorithm  corrects  I  in  three  main  phases.  First,  the  algorithm  checks  whether  I 
is  symmetric  with  diagonal  entries  of  1 .0  and  off-diagonal  entries  between  [-1 ,  1],  If  it  is  not, 
the  user  is  prompted  to  re-input  the  matrix,  correcting  for  the  discrepancies. 

Second,  once  the  above  conditions  are  satisfied,  the  algorithm  checks  whether  I  is 
positive-definite.  One  way  to  test  this  is  to  find  the  eigenvalues  for  I  (positive-definite 
matrices  have  all  positive  eigenvalues).  The  paper  referenced  did  not  describe  an  approach 
for  finding  the  eigenvalues  of  the  matrix.  After  further  research,  the  Jacobi  Eigenvalue 
Algorithm  was  determined  to  be  a  sufficiently  efficient  way  to  evaluate  a  matrix’s 
eigenvalues.  As  a  result,  the  eigenvalues  are  produced  as  the  diagonals  of  an  otherwise 
zero-matrix.  The  Jacobi  Eigenvalue  Algorithm  is  computationally  inexpensive  and  pre¬ 
existing  source  code  was  used  in  its  implementation. 

If  all  eigenvalues  for  I  are  positive  and  the  other  conditions  have  been  satisfied,  then 
I  is  a  consistent  correlation  matrix.  Otherwise,  in  the  third  phase,  negative  eigenvalues  are 
changed  to  small,  positive  values  (e.g.,  .000001).  The  diagonal  matrix  of  adjusted 
eigenvalues  is  then  multiplied  by  the  associated  matrix  of  eigenvectors  (also  produced  using 
the  Jacobi  Eigenvalue  Algorithm).  That  product  is,  in  turn,  multiplied  by  the  inverse  of  the 
matrix  of  eigenvectors  to  arrive  at  a  new  matrix  that  is  similar,  but  not  equal  to,  I.  Lastly,  the 
diagonals  are  reset  to  1.0  as  they  may  have  changed  during  the  transformation.  This  third 
section  of  the  algorithm  is  repeated  until  all  eigenvectors  of  the  adjusted  matrix  are  positive. 
At  this  point,  the  user  input  matrix  has  been  adjusted  such  that  it  is  a  consistent  correlation 
matrix. 

Though  the  User-Input  Correlation  Matrix  is  now  a  consistent  correlation  matrix, 
the  transformation  of  I  is  not  complete  and  the  Adjusted  Correlation  Matrix  (L)  has  not 
been  determined.  As  will  be  shown  later,  depending  on  the  parent  distributions  being 
correlated,  there  may  be  a  maximum  achievable  correlation  between  any  two  of  the 
variables.  Determination  of  L  will  be  covered  later  in  the  section:  Correcting  the  User-Input 
Correlation  Matrix  (Part  II). 
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Correlating  Input  Random  Variables 

In  order  to  understand  how  the  Applied  Correlation  Matrix  (A)  is  to  be  optimized 
such  that  the  Output  Correlation  Matrix  (O)  is  identical  to  the  Adjusted  Correlation 
Matrix  (L),  the  method  for  correlating  the  parent  distributions  must  first  be  discussed.  It  is  a 
well-known  fact  that  normal  random  variables  can  be  correlated  by  multiplying  a  vector  of 
uncorrelated  normal  random  draws  by  the  Cholesky  decomposition36  of  the  desired 
correlation  matrix.  The  Lurie-Goldberg  Method  takes  this  one  step  further  using  normal 
random  variates  to  generate  correlated  uniform  random  variates.  These  uniform  random 
variates  are  then  transformed  via  the  inverse-CDF  technique  to  generate  draws  from  the 
desired  parent  distributions.  In  this  method,  although  the  correlations  between  the  normal 
random  draws  are  known,  as  these  draws  are  transformed  into  other  distributions,  the 
correlations  change.  Hence,  the  core  problem  emerges:  how  can  the  Optimal  Applied 
Correlation  Matrix  (A’)  be  uncovered  such  that  O  matches  L?  Answering  this  question  is 
key  to  implementing  the  Lurie-Goldberg  Method.  The  authors  have  developed  an  algorithm 
that  addresses  this  very  question,  without  necessitating  any  runs  of  the  simulation. 
Additionally,  they  have  begun  the  process  of  optimizing  this  algorithm,  finding  heuristics  that 
allow  it  to  run  with  a  minimal  number  of  calculations. 

Implementation  and  Application  of  the  CCA 

The  CCA’s  chief  advantage  is  that  it  is  non-recurring  and  its  implementation  requires 
no  simulation.  Furthermore,  because  the  algorithm  only  requires  looking  at  pairs  of  parent 
distributions,  once  the  applied  matrix  has  been  found  for  a  set  of  parent  distributions,  the 
algorithm  must  only  be  run  when  distributions  are  added  or  changed,  and  even  then,  only  for 
the  new/altered  distributions.  The  algorithm  also  uses  Pearson’s  correlation  while  COTS  risk 
tools  substitute  Spearman’s  rank  correlation. 

The  applications  of  the  CCA  reach  beyond  the  Cost  and  Risk  analysis  community; 
this  algorithm  is  useful  anywhere  there  is  a  need  to  induce  Pearson’s  correlation  between 
input  variables.  For  example,  this  algorithm  can  applied  to  auto  correlating,  stock  market 
projections  in  the  financial  arena  and  to  traditional  modeling  and  simulation  situations  when 
correlation  is  needed.  The  algorithm  was  designed  with  a  focus  on  portability.  Because 
algorithm  is  coded  with  Visual  Basic,  it  can  be  easily  integrated  in  existing  tools  and  models. 
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1.  Introduction 

The  US  Department  of  Defense  (DoD)  is  in  the  process  of  radical  transformation — 
transformation  to  a  national  security  strategy  predicated  on  joint  Service  purchases  and 
complex  System-of-Systems  (SoS)  capabilities.37  This  paper  contributes  to  a  broader  study 
that  eventually  needs  to  be  conducted  to  evaluate  the  benefits  and  costs  of  increased 
reliance  on  joint  Service  SoS  programs. 

The  DoD’s  increasing  emphasis  on  joint  Service  SoS  capabilities  has  created  both 
opportunities  and  challenges  for  materiel  acquisition.  In  terms  of  improving  the  effectiveness 
of  warfighting  capabilities,  the  opportunity  exists  for  joint,  interoperable,  multi-function,  multi¬ 
mission  systems  that  leverage  information  dominance  and  improve  decisions  and  outcomes 
by  making  US  and  coalition  forces  not  only  better  informed,  but  more  coordinated,  faster 
and  more  adaptive.  In  terms  of  efficiencies,  multiple  opportunities  exist  for  joint  programs  to 
cut  “economic  production  costs” — for  instance,  by  reducing  duplication,  by  exploiting 
learning  curves,  and  by  achieving  economies  of  scale  and  scope  in  manufacturing,  and  in 
operations  and  support  activities  (e.g.,  joint  training  and  logistics). 

However,  there  is  a  dark  side.  A  key  barrier  needs  to  be  overcome  for  the  DoD  to 
achieve  the  promises  of  joint  Service  SoS  programs.  This  involves  the  challenge  of 
“transaction  (coordination  and  motivation)  costs.”  These  are  the  less  visible,  but  nonetheless 
significant,  costs  of  negotiating,  managing  and  monitoring  transactions. 

There  are  a  variety  of  categories  of  joint  Service  programs.  These  range  from 
relatively  simple,  single-system,  single-Service  programs  to  which  other  Services  sign  on  to 
use  the  end  product,  to  fully  integrated,  multi-Service  SoS  programs.  Examples  of  the  latter 
include  the  Joint  Strike  Fighter  (JSF)  and  the  Joint  Surveillance  Target  Attack  Radar  System 
(JSTARS).  Clearly,  the  latter  (joint  Service  SoS)  acquisitions  are  considerably  more 
challenging  than  the  former.  The  reason  can  be  traced  to  greater  interdependence, 
manifested  in  greater  complexity  and  uncertainty. 


37  As  defined  in  the  DoD  Defense  Acquisition  Guidebook  (2004),  an  SoS  is  “a  set  or  arrangement  of 
systems  that  results  when  independent  and  useful  systems  are  integrated  into  a  larger  system  that 
delivers  unique  capabilities.”  Joint  Program  Management  is  defined  as  “Any  defense  acquisition 
system  [. . .]  or  technology  program  that  involves  formal  management  or  funding  by  more  than  one 
DoD  Component  during  any  phase  of  a  system’s  life  cycle”  (DAU,  2004,  p.1 ).  DoD  Directive  5000. 1, 
The  Defense  Acquisition  System,  dated  12  May  2003,  indicates  a  policy  preference  for  joint 
development  programs  over  Component-unique  development  programs  (USD(AT&L). 
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Joint  Service  SoS  acquisitions  involve  more  stakeholders,  multiple  users  and  funding 
sources,  divergent  and  competing  requirements,  and  conflicting  objectives  that  lead  to 
difficult  and  contentious  tradeoffs,  diffused  authority,  negotiated  budget  arrangements, 
complex  project  management  structures,  etc.  This  increased  interdependence  is  generally 
reflected  in  greater  transaction  costs:  higher  “coordination  costs”  from  increased  complexity 
and  uncertainty,  and  higher  “motivation  costs”  from  greater  asset  specificity  and  limited 
market  contestability. 

The  higher  the  transaction  costs,  the  greater  economic  production  cost  efficiencies 
need  to  be  to  offset  them.  As  recently  emphasized  by  Melese,  Franck,  Angelis  and  Dillard 
(2007),  if  initial  cost  estimates  focus  exclusively  on  economic  production  costs  and  ignore 
transaction  costs,  the  result  can  be  systematic  cost  overruns. 

According  to  a  recent  RAND  study  of  major  acquisition  programs,  the  average  cost 
overruns  of  weapon  systems  in  the  development  phase  ranged  from  16%  to  26%. 
Procurement  cost  growth  over  initial  estimates  averaged  between  16%  and  65%,  while  total 
weapon  program  cost  overruns  averaged  from  20%  to  54%  (Arena,  Leonard,  Murray  & 
Younossi,  2006). 

Since  the  official  “acquisition  program  baseline”  (APB)  estimates  for  these  programs 
reflect  the  best  current  understanding  of  the  cost,  schedule,  and  performance  objectives  at 
the  time  the  baseline  is  established  (typically  at  Milestone  B  decisions),  Congressional 
funding  of  these  programs  represents  an  implicit  contract  with  the  Executive  Branch.  When 
incomplete  or  unrealistic  cost  estimates  lead  to  significant  cost  and  schedule  overruns 
relative  to  expectations  established  in  the  APB,  administrative  sanctions,  such  as  statutory 
(Nunn-McCurdy38)  unit  cost  breaches,  can  be  triggered.  In  turn,  these  breaches  can 


38  Since  the  law  was  enacted  in  1982,  Title  10  USC  Section2433,  a  “Nunn-McCurdy”  unit  cost  breach 
occurs  when  a  major  defense  acquisition  program  experiences  an  increase  of  at  least  15%  in 
program  acquisition  unit  cost  or  average  procurement  unit  cost  above  the  unit  costs  in  the  acquisition 
program  baseline.  Through  2006,  the  DoD  had  the  ability  to  administratively  change  the  acquisition 
program  baseline  for  the  purposes  of  unit  cost  reporting,  and  so  was  able  to  reduce  the  number  of 
apparent  Nunn-McCurdy  breaches,  despite  apparent  cost  growth  that  would  otherwise  trigger  the 
Nunn-McCurdy  sanctions.  In  2006,  the  Nunn-McCurdy  law  was  amended.  The  FY  2006  National 
Defense  Authorization  Act  ( NDAA )  severely  restricted  the  DoD’s  ability  to  change  unit  cost  reporting 
criteria.  As  a  result  of  this  change,  Nunn-McCurdy  unit  cost  breaches  are  incurred  at  15% 
(“Significant”),  and  25%  (“Critical”)  above  the  current  unit  cost  threshold.  Additionally,  a  “Significant” 
Nunn-McCurdy  breach  is  incurred  at  30%  of  the  Milestone  B  unit  cost  threshold,  and  a  “Critical” 
Nunn-McCurdy  breach  is  declared  at  50%  growth  above  the  Milestone  B  unit  cost  threshold.  Thus, 
the  ability  of  the  DoD  to  mask  unit  cost  growth  through  changes  to  unit  cost  thresholds  is  restricted. 
The  sanctions  imposed  by  Congress  on  programs  breaching  Nunn-McCurdy  criteria  are  noteworthy: 
For  “Significant”  breaches,  the  Service  Secretary  must  notify  Congress  within  45  days  of  the  report 
(normally  program  deviation  report)  upon  which  the  determination  is  based  (normally  a  program 
deviation  report  initiated  when  the  Program  Manager  becomes  aware  of  the  breach).  The  program 
must  submit  a  Selected  Acquisition  Report  (SAR)  with  the  required  unit  cost  breach  information. 

For  “Critical”  breaches,  the  Defense  Secretary  (usually  delegated  to  the  Under  Secretary  of  Defense 
for  Acquisition,  Technology,  and  Logistics)  must  certify  to  Congress  within  60  days  of  notification  that: 

1 )  the  program  is  essential  to  national  security, 

2)  there  is  no  alternative  which  can  provide  equal  capability  at  less  cost, 

3)  the  updated  estimates  of  unit  cost  (calculated  independently  by  the  OSD  Cost  Analysis 
Improvement  Group)  are  reasonable,  and 

4)  the  management  structure  is  adequate  to  control  unit  cost  going  forward. 
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dramatically  impact  program  execution;  they  also  jeopardize  relations  between  the 
Legislative  and  Executive  Branches  of  government. 

A  study  conducted  by  the  DoD  in  2007  to  develop  a  business  case  for  improving 
system  cost  estimating  in  the  DoD  (Brown,  Flowe  &  Hamel,  2007b)  examined  the  cost 
growth  of  all  major  defense  acquisition  programs  (MDAPs)  from  1995  to  2005  to  determine 
the  source  and  relative  magnitude  of  cost  growth  and  schedule  breaches.  See  Figure  1 . 


Ten  Year  Cost  Change  by  Category 
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Figure  1.  Cost  Growth  by  Category  for  All  MDAPs  1995-2005 

(Brown,  Flowe  &  Hamel,  2007b) 

As  shown  in  Figure  1 ,  the  largest  source  of  cost  growth  as  reported  in  the  Selected 
Acquisition  Reports  (SAR)  is  “Estimation.”  This  indicates  that  estimates  made  in  establishing 
MDAP  acquisition  program  baselines  were  often  in  error,  and  thus  program  costs  appeared 
to  grow  despite  very  little  change  in  the  objective  content  of  the  program.  This  “Estimation” 
error  accounted  for  approximately  $201  billion  in  apparent  cost  growth  over  the  10-year 
period  examined  across  all  major  programs. 

Figure  2  is  a  related  analysis  that  examines  the  quantity  and  sources  of  breaches  for 
108  MDAPs  over  the  same  10-year  period  (1995-2005).  With  respect  to  the  Acquisition 
Program  Baseline  (APB),  schedule  breaches  were  the  most  common;  occurring  244  times 


Failure  to  certify  within  the  60-day  timeframe  will  result  in  suspension  of  obligations  for  major 
contracts  until  30  days  of  continuous  session  of  Congress,  beginning  from  the  date  of  receipt  of 
SAR/Certification. 
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over  the  10-year  period  examined,  suggesting  that  multiple  schedule  breaches  is  a  relatively 
common  occurrence  in  many  programs. 


Breach  Incidence  by  Type 


□  Breach  Count 
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Figure  2.  Breaches  Reported  by  SAR  All  MDAPs  1995-2005 

(Brown,  Flowe  &  Hamel,  2007b) 

Development  and  procurement  cost  breaches  occur  with  less  frequency  than 
schedule  breaches,  but  still  occur  sufficiently  frequently  that  on  average,  each  MDAP  can 
expect  to  have  one  of  each.  Program  Acquisition  Unit  Cost  (PAUC)  breaches  occur  nearly 
as  frequently  as  schedule  breaches,  indicating  that  the  confluence  of  development  and 
procurement  cost  increases  conspire  to  increase  unit  costs  at  least  10%  above  that 
established  by  the  APB.  Nunn-McCurdy  breaches  are  notably  less  common,  suggesting  that 
only  about  20%  of  programs  that  breach  their  APB  unit  costs  will  grow  substantially  beyond 
that.  Overall,  the  raw  quantity  of  breaches  indicate  that  expectations  regarding  costs  and 
schedules  are  usually  unmet,  to  the  extent  that  for  the  108  MDAPs  examined,  each 
breached  on  average  more  than  twice  for  schedule  and  unit  cost,  and  at  least  once  for 
development  cost.39 

This  paper  investigates  cost  and  schedule  breaches  in  a  subset  of  MDAPs  that 
includes  a  sample  of  84  programs,  divided  into  “Joint  Service”  and  “Traditional”  (single 
Service)  acquisition  programs,  and  “Single  System”  and  “System-of-Systems”  (SoS) 
programs.  (The  data  is  available  upon  request  from  the  authors.)  The  results  reported  in 
Appendix  I  suggest  there  is  a  statistically  significant  higher  risk  of  cost  and  schedule 
breaches  in  SoS  programs  than  in  single  system  acquisition  programs.  Interestingly,  while 


39  This  is  not  to  imply  that  every  program  was  equally  troubled.  A  subset  of  particularly  troubled 
programs,  approximately  30%  of  the  total,  breached  in  some  way  every  year  they  reported. 
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“Joint  Service”  programs  in  general  have  a  weak  statistically  significant  greater  risk  of 
schedule  (and  RDT&E  cost)  breaches  than  “Traditional”  (single  Service)  programs,  there  is 
no  significant  difference  between  the  two  in  terms  of  Program  Acquisition  Unit  Cost  (PAUC) 
breaches  or  Average  Procurement  Unit  Cost  (APUC)  breaches.  (See  Tables  1  &  2  in 
Appendix  I.) 

Based  on  our  sample,  SoS  programs  tend  to  take  relatively  longer  and  cost  more 
than  single  system  acquisitions.  This  preliminary  empirical  evidence  of  cost  and  schedule 
breaches  suggests  initial  cost  (schedule)  estimates  of  SoS  programs  may  not  be  adequately 
capturing  transaction  costs.  In  fact,  since  production  cost  breaches  are  significantly  greater, 
the  transaction  costs  experienced  by  SoS  programs  may  be  overwhelming  any  potential 
economic  production  cost  savings  (Melese  et  al.,  2007).  If  this  is  indeed  the  case,  then  the 
anticipated  warfighting  benefits  of  SoS  solutions  need  to  be  sufficiently  large  to  compensate 
for  the  extra  costs  and  schedule  delays  experienced  by  these  programs. 

Meanwhile,  it  appears  that  while  “Jointness”  contributes  to  schedule  overruns,  it  only 
weakly  contributes  to  development  cost  overruns,  and  by  itself,  does  not  explain  production 
cost  overruns.  While  our  results  suggest  joint  programs  tend  to  breach  their  schedules 
relatively  more  often  than  single  Service  programs  (“every  event  in  a  joint  program  takes 
longer  [...]  extra  time  needs  to  be  included  in  the  program  schedule”  (DAU,  2004,  p.  20)), 
joint  programs  in  our  sample  only  experienced  a  few  more  cost  breaches  than  single 
Service  programs  in  the  early  development  (RDT&E)  stages.  There  does  not  appear  to  be 
any  statistical  difference  in  production  cost  breaches  (PAUC  or  APUC)  between  joint  Service 
and  single  Service  programs. 

One  explanation  for  this  is  that  the  joint  Service  programs  in  the  sample  encompass 
a  spectrum  that  includes  both  single  systems  and  SoS,  and  that  the  complexity  and 
uncertainty  associated  with  SoS  is  so  significant  as  to  overwhelm  any  additional  complexity 
and  uncertainty  that  might  be  experienced  by  joint  Service  programs.  Another  possibility  is 
that  built-in  checks  and  balances  tend  to  offset  the  extra  transaction  costs  of  joint  Service 
programs. 

For  instance,  “joint  programs  require  special  attention  to  multi-service  funding 
requirements  and  to  acquiring  the  right  mix  of  joint  expertise  for  the  source  selection 
process”;  indeed,  “full  consultation  and  coordination  with  the  participating  components”  is 
required  (DAU,  2004,  pp.  12,  21).  The  ultimate  outcome  may  be  to  help  anticipate  and 
mitigate  the  extra  transaction  costs,  and  to  avoid  requirements  creep  in  production  stages  of 
the  program.  Guidance  for  joint  program  managers  is  designed  to  inhibit  any  changes  in 
scope,  stating  that  “substantive  changes  to  [...]  program  documentation,  such  as  the 
acquisition  strategy  or  the  contract  [need  to  be]  negotiated  with  the  participating 
Components  prior  to  making  changes”  (p.  12). 

With  weapon  system  investments  expected  to  capture  a  significant,  and  perhaps 
growing,  share  of  defense  budgets,  unprecedented  attention  has  been  devoted  to  clarifying 
the  determinants  of  risk,  failure,  and  success  in  the  joint  arena  (Pracchia,  2004).  The 
defense  department’s  apparent  inability  to  avert  or  even  predict  adverse  program  outcomes 
such  as  cost  and  schedule  breaches  is  not  only  a  source  of  external  criticism  (GAO,  2006) 
and  internal  attention  (Krieg,  2005),  it  has  undermined  confidence  in  the  time-honored 
practices  of  program  management  and  oversight. 
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To  date,  there  is  significant  debate  regarding  the  factors  that  influence  the  outcomes 
of  programs.  Thus,  the  search  for  root  causes  and  potential  solutions  of  program  cost 
growth,  schedule  delay,  and  capability  shortfall  have  received  increased  attention.  To  help 
explain  potential  pitfalls  associated  with  joint  programs,  this  study  leverages  “Transaction 
Cost  Economics”  (TCE),  which  has  recently  been  applied  to  generate  new  insights  into 
defense  cost  overruns  (Melese  et  al.,  2007). 

2.  System-of-Systems  (SoS)  and  a  Declining  Acquisition 
Workforce 

In  2003,  the  Joint  Capabilities  Integration  and  Development  System  (JCIDS) 
replaced  the  “Requirements  Generation  System”  that  had  identified  warfighter  needs  for 
nearly  30  years  (recently  updated  in  CJCS,  2007,  May  1).  Providing  a  new,  substantive  role 
for  Combatant  Commanders,  the  JCIDS  process  reflects  a  significant  shift  in  the  focus  of 
defense  programming  toward  joint  system  capabilities.  But  an  emerging  concern  is  that  DoD 
technology  investments  are  becoming  increasingly  concentrated  in  very  large,  very  complex 
system-of-systems. 

In  several  recent  cases,  the  size  and  complexity  of  undertaking  SoS  programs  has 
overwhelmed  the  DoD’s  ability  to  effectively  manage  them.  According  to  a  recent 
Congressional  Research  Service  report,  “management  and  oversight  of  acquisition 
programs  increases  as  the  value  of  the  program  increases”  (Chadwick,  2007,  p.  12).  The 
larger  the  program,  “the  more  difficult  it  is  to  sustain  communications  among  staff  members. 
In  general,  if  there  are  “n”  people  in  a  program,  the  potential  number  of  pair-wise  channels  is 
n(n-1)/2  [...Ljarger  teams  [...]  have  a  greater  chance  of  communications  breakdown”  (DAU, 
2004,  p.  16).  Ceteris  paribus,  the  bigger  the  program,  the  larger  the  transaction 
(coordination)  costs. 

Compounding  the  challenge  was  a  steep  decline  in  the  professional  acquisition 
workforce  available  to  negotiate,  manage,  monitor  and  enforce  contracts.  The  post-Cold 
War  draw-down,  and  the  Acquisition  Reform  initiatives  of  the  mid  1990s  saw  dramatic 
declines  in  the  defense  acquisition  workforce,  with  a  corresponding  “brain  drain  of  organic 
government  program  management  capability,  particularly  in  specialized  acquisition-related 
fields  such  as  systems  engineering.”40  The  combination  of  the  dramatically  increased  size 


40  The  consequences  of  this  “brain  drain”  have  been  noted  by  Congress,  such  that  HR1585  (bill  as 
passed  by  the  House  and  Senate)  FY08  National  Defense  Authorization  Act  SEC.  852. 
DEPARTMENT  OF  DEFENSE  ACQUISITION  WORKFORCE  DEVELOPMENT  FUND  stated  as 
follows: 

— Chapter  87  of  title  10,  United  States  Code,  is  amended  by  inserting  after  section  1704  the 
following  new  section:  “§  1705.  Department  of  Defense  Acquisition  Work  force  Development 
Fund”  (a)  ESTABLISHMENT. — The  Secretary  of  Defense  shall  establish  a  fund  to  be  known 
as  the  “Department  of  Defense  Acquisition  Workforce  Fund”  (in  this  section  referred  to  as  the 
“Fund”)  to  provide  funds,  in  addition  to  other  funds  that  may  be  available,  for  the  recruitment, 
training,  and  retention  of  acquisition  personnel  of  the  Department  of  Defense. 

(b)  PURPOSE. — The  purpose  of  the  Fund  is  to  ensure  that  the  Department  of  Defense 
acquisition  workforce  has  the  capacity,  in  both  personnel  and  skills,  needed  to  properly 
perform  its  mission,  provide  appropriate  oversight  of  contractor  performance,  and  ensure  that 
the  Department  receives  the  best  value  for  the  expenditure  of  public  resources.” 
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and  complexity  of  SoS  programs  coupled  with  a  reduced  DoD  acquisition  workforce  offers 
one  explanation  for  adverse  programmatic  outcomes  that  can  be  informed  by  TCE  (Krieg, 
2005). 


To  compensate  for  its  declining  internal  expertise,  the  DoD  devised  and  increasingly 
relied  upon  external  “Lead  Systems  Integrators”  to  manage  complex  programs.  Based  on 
several  recent,  highly  publicized  failed  attempts  to  outsource  the  integration  of  complex 
systems  to  private  “Lead  Systems  Integrators”  (LSI),  Congress  passed  HR  1585  for  2008 
defense  authorizations,  which  formally  issues  a  “Prohibition  on  the  Use  of  Lead  Systems 
Integrators”  (Sec.  802).  This  effectively  brings  what  are  considered  to  be  “inherently 
governmental”  responsibilities  (such  as  Lead  Systems  Engineer,  Lead  Cost  Analyst, 
Program  Manager,  Deputy  Program  Manager,  Systems  Architect)  back  into  the  public 
domain  by  201 1.41 

To  the  extent  the  foregoing  legislation  drives  the  responsibilities  and  functions  back 
into  the  public  sector  (versus  simply  reasserting  Government  oversight),  it  bears  noting  that 
vertically  integrating  the  responsibility  for  systems  integration  within  the  DoD  is  not  without 


41  FY07  NDAA  SEC.  820.  GOVERNMENT  PERFORMANCE  OF  CRITICAL  ACQUISITION  FUNCTIONS. 

(a)  Goal. — It  shall  be  the  goal  of  the  Department  of  Defense  and  each  of  the  military 
departments  to  ensure  that,  within  five  years  after  the  date  of  the  enactment  of  this  Act,  for  each 
major  defense  acquisition  program  and  each  major  automated  information  system  program,  each  of 
the  following  positions  is  performed  by  a  properly  qualified  member  of  the  Armed  Forces  or  full-time 
employee  of  the  Department  of  Defense: 

(1)  Program  manager. 

(2)  Deputy  program  manager. 

(3)  Chief  engineer. 

(4)  Systems  engineer. 

(5)  Cost  estimator. 

(b)  Plan  of  Action. — Not  later  than  six  months  after  the  date  of  enactment  of  this  Act,  the 
Secretary  of  Defense  shall  develop  and  begin  implementation  of  a  plan  of  action  for  recruiting, 
training,  and  ensuring  appropriate  career  development  of  military  and  civilian  personnel  to  achieve 
the  objective  established  in  subsection  (a).  The  plan  of  action  required  by  this  subsection  shall 
include  specific,  measurable  interim  milestones. 

(c)  Reports. — Not  later  than  one  year  after  the  date  of  the  enactment  of  this  Act  and  each 
year  thereafter,  the  Secretary  of  Defense  shall  submit  to  the  congressional  defense  committees  a 
report  on  the  progress  made  by  the  Department  of  Defense  and  the  military  departments  toward 
achieving  the  goal  established  in  subsection  (a). 

(d)  Definitions. — In  this  section: 

(1 )  The  term  “major  defense  acquisition  program”  has  the  meaning  given  such  term  in  section 
2430(a)  of  title  10,  United  States  Code. 

(2)  The  term  “major  automated  information  system  program”  has  the  meaning  given  such 
term  in  section  2445a(a)  of  title  10,  United  States  Code  (as  added  by  section  816  of  this  Act). 
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its  own  set  of  risks.  Transaction  cost  economics  (TCE)  suggests  vertical  integration  leads  to 
internal  bureaucratic  coordination  and  motivation  issues.  These  include  the  risk  of  internal 
opportunistic  behavior  (costly  lobbying  for  promotions  or  budgets),  multi-tasking  (“what  gets 
measured  gets  done”),  and  sub-optimization  (success  achieved  at  lower  levels  at  the 
expense  of  the  overall  success  of  the  program). 

To  mitigate  these  internal  transaction  costs,  Kelman  (2005)  emphasizes  three  key 
features  of  a  “bureaucratic  organization”:  “the  extensive  use  of  rules,  hierarchy,  and 
specialization.”  (p.  10).  DoD  procurement  activities  are  governed  by  several  sets  of  federal 
government  rules/regulations.  For  contracting,  the  Federal  Acquisition  Regulation  {FAR) 
applies  to  the  entire  federal  government.  The  Defense  FAR  Supplement  (DFARS)  applies 
only  to  the  DoD.  A  subordinate  set  of  governing  regulations,  outlined  in  Component-unique 
FAR  Supplements,  apply  to  individual  DoD  Components.  Still  another  set  of  regulations,  the 
DoD  5000  series,  pertains  to  the  acquisition  of  materiel —  derived  from  the  1977  OMB 
Circular  A-1 09,  Major  System  Acquisitions.  Another  layer  of  complexity  is  imposed  by 
several  provisions  within  Titles  10  and  40  of  the  US  Code,  which  impose  specific  additional 
requirements  for  the  procurement  of  information  technology  in  general,  and  business 
systems  in  particular42. 

While  these  bureaucratic  structures  are  imposed  to  address  transaction  costs,  some 
argue  the  “cure  is  worse  than  the  disease,”  in  that  the  layers  of  often  obscure  and  conflicting 
guidance  introduce  new  transaction  costs  beyond  what  would  have  been  experienced  in 
their  absence.  These  transaction  costs  can  be  experienced  as  delays  in  procuring  needed 
operational  capability,  in  which  case  the  “discipline”  of  the  regulated  bureaucracy  translates 
into  a  lack  of  agility  that  imposes  “time  to  market”  penalties. 

In  light  of  the  competing  requirements  for  governance  mechanisms  to  mitigate 
transaction  costs — arising  from  vertical  integration  versus  requirements  for  acquisition  agility 
and  the  dramatically  increasing  complexity  of  large-scale  systems — acquisition  policies  and 
procedures  are  undergoing  intense  scrutiny  for  their  ability  to  support  joint  capabilities  (DAU, 
2004,  Chapter  1,  “Joint  Program  Management  Introduction”).  Many  other  changes  are 
underway  in  the  acquisition  arena.  Recognizing  that  single  system  acquisition  methods  may 
not  readily  apply  to  joint  SoS  capability-based  acquisition  efforts,  the  search  is  on  for  a 
clearer  understanding  of  how  various  acquisition  strategies  either  support  or  impede  joint 
efforts.  Transaction  Cost  Economics  (TCE)  offers  a  valuable  framework  to  guide  the  way 
forward. 


42  Notably,  the  Clinger  Cohen  Act  of  1996,  40  USC  §1401  et  seq.,  and  10  USC  §  2222,  Defense 
Business  Systems:  Architecture,  Accountability,  and  Modernization,  establish  specific  requirements 
for  procurement  of  information  technology  that,  in  several  key  respects,  differ  from  the  procurement  of 
weapon  systems  and  other  major  defense  acquisition  programs,  the  responsibility  for  which  is 
established  by  10  USC  §133,  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics. 
The  Clinger-Cohen  Act  effectively  establishes  a  separate  “chain  of  command”  for  IT  acquisition 
programs,  which  runs  from  the  Director  of  the  Office  of  Management  and  Budget  to  the  DoD  Chief 
Information  Officer  (CIO),  established  by  44  USC  §3506  as  the  “Senior  official  of  the  Department  of 
Defense”  for  IT  matters. 
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3.  Transaction  Costs  =  Motivation  Costs  +  Coordination  Costs 

In  business,  two  costs  are  typically  factored  into  strategic  acquisition  decisions: 
production  costs  and  the  costs  of  managing  the  transaction — or  “transaction  costs”  (Coase, 
1937).  Conventional  strategies  tend  to  focus  on  economic  production  costs  (input  costs, 
learning  curves,  economies  of  scale  and  scope,  etc.).  TCE  emphasizes  another  set  of 
costs — coordination  and  motivation  costs.  Economic  production  (opportunity)  cost 
advantages  tend  to  guide  companies  to  specialize  in  “core”  activities  in  which  they  have  a 
comparative  advantage,  and  to  “transact”  with  outside  suppliers  to  acquire  other  goods  and 
services.  A  key  contribution  of  TCE  is  to  formally  introduce  and  fully  reveal  the  nontrivial 
costs  of  managing  those  transactions. 

Transaction  costs  include  coordination  and  motivation  costs,  such  as  search  and 
information  costs,  decision,  contracting,  and  incentive  costs,  and  measurement,  monitoring, 
and  enforcement  costs.  TCE  predicts  these  costs  will  vary  across  weapon  system 
acquisition  programs  to  the  extent  there  are  differences  in  certain  key  characteristics  of  the 
transaction — complexity,  uncertainty,  frequency,  and  asset  specificity — as  well  as  market 
contestability,  and  the  choice  of  governance  (contracting,  etc.)  mechanisms.  A  central  point 
from  TCE  is  that  the  choice  of  contract,  organization,  and  incentives,  along  with  key 
characteristics  of  the  transaction  (complexity,  uncertainty,  frequency,  and  asset  specificity) 
and  market  contestability,  must  be  considered  in  order  to  obtain  reliable  cost  estimates  of 
joint  programs  (Melese  et  al.,  2007). 

One  of  the  key  insights  of  TCE  is  that  capital  (and  human  capital  investments)  that 
are  specific  to  a  transaction  (e.g.,  made  to  support  a  joint  program  acquisition)  can  generate 
cost  savings,  but  also  carry  the  risk  of  increasing  transaction  costs  from  opportunistic 
behavior.  The  role  of  relation-specific  investments  (“asset  specificity”)  is  an  important 
consideration  that  needs  to  be  anticipated  and  factored  into  any  analysis  of  joint  Service 
SoS  programs.43 

4.  Motivation  Costs:  The  Role  of  Asset  Specificity 

Having  made  a  specialized  investment  in  location,  physical,  human,  or  other  specific 
assets,  a  supplier  often  becomes  the  most  efficient  provider,  which  is  good  from  a 
production-cost  perspective,  but  provides  incentives  for  the  supplier  to  look  for  opportunities 
to  extract  more  from  the  transaction  (perhaps  by  demanding  steep  prices  for  any 
modifications  to  the  contract).  After  investments  in  specific  assets  are  made,  the  relationship 
is  transformed  from  a  customer  having  the  choice  of  a  number  of  competing  suppliers  to  a 
bilateral  monopolistic  relationship  between  a  buyer  and  seller. 


43  Williamson  (1996)  identifies  six  types  of  asset  specificity:  1)  site,  2)  physical  asset,  3)  human  asset, 
4)  dedicated  asset,  5)  brand-name  capital,  and  6)  temporal.  These  are  specialized  investments  that 
generate  high  returns  within  a  specific  relationship,  but  offer  little  value  outside  it.  Site  specificity 
refers  to  the  co-location  of  facilities  to  minimize  inventory  or  production  costs.  Physical  asset 
specificity  refers  to  the  use  of  customized  assets  such  as  specialized  dies  and  equipment.  Human 
asset  specificity  refers  to  firm-specific  knowledge  and  skills  (e.g.,  “specific”  as  opposed  to  “general” 
training).  Dedicated  asset  specificity  refers  to  additional  investments  in  plant  and  equipment  made  to 
sell  the  extra  output  to  a  specific  customer.  Brand-name  capital  specificity  refers  to  investments  in 
reputation.  Temporal  asset  specificity  refers  to  investments  that  facilitate  timing  and  coordination  of 
projects  (e.g.,  investments  in  critical-path  activities). 
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Similar  to  “sunk  costs,”  investments  in  relationship-specific  assets  (“asset  specificity”) 
are  potentially  valuable,  but  can  increase  risks  to  both  parties  in  a  transaction  (Klein, 
Crawford  &  Alchian,  1978).  Close-in  bilateral  bargaining  (a  principal-agent  type  game) 
replaces  the  competitive  marketplace.  This  entails  a  transformation  of  the  supplier  from 
competitive  bidder  (prior  to  source  selection)  to  monopoly  supplier  (after  source  selection), 
especially  if  there  are  no  close  substitutes.  Accordingly,  the  customer  (government)  is  now 
vulnerable  to  “opportunistic  behavior”  from  the  supplier. 

Unforeseen  contingencies,  combined  with  newly  inelastic  demand,  may  prompt  the 
supplier  to  extract  more  of  the  surplus  created  in  the  relationship.44  In  this  case,  suppliers 
can  exploit  their  power  in  the  relationship  by  renegotiating  a  basic  agreement  to  their 
advantage,  otherwise  threatening  to  dissolve  the  agreement.  The  TCE  literature  refers  to 
this  as  a  “hold-up”45  (Klein,  Crawford  &  Alchian,  1978). 

Conversely,  a  supplier  (defense  contractor)  that  makes  specific  investments  in 
assets  that  are  only  valuable  in  the  context  of  the  relationship  with  a  specific  customer 
(government),  can  find  itself  vulnerable  to  any  changes  in  demand  from  that  customer  (i.e., 
the  supplier  suffers  from  “demand  uncertainty”).  Given  the  government  is  the  only  buyer  (or 
one  of  only  a  few)  of  joint  SoS  weapon  systems,  and  given  its  limited  ability  to  commit  as  a 
result  of  the  annual  nature  of  most  budgetary  processes,  defense  industry  sellers  often  face 
a  monopsony  buyer  that  cannot  make  credible  multi-year  commitments.  This  leads  to  sellers 
facing  substantial  demand  uncertainty  and  the  real  risk  of  strategic  renegotiation. 

Whereas  relation-specific  investments  can  increase  the  total  gains  to  both  parties, 
the  risk  exists  of  opportunistic  behavior;  either  party  can  hold  up  the  other,  for  instance,  by 
threatening  to  change  the  terms  of  the  contract  (e.g.,  the  government’s  sovereign  right  to 
terminate  a  contract  for  convenience  as  well  as  default).  The  danger  is  that  if  neither  party 
feels  it  can  recover  the  full  costs  of  its  investment  in  the  relationship/transaction  (say  through 


44  Williamson  (1975),  Besanko,  Dranove,  Shanley  and  Schaefer  (2000),  and  others  have  labeled  the 
transition  from  one  prospective  buyer  and  many  sellers  to  one  buyer  and  one  seller,  or  from 
competitive  market  to  bilateral  monopoly,  as  the  “fundamental  transformation.”  To  some  extent,  this 
transformation  occurs  after  the  completion  of  every  military  source-selection  process. 

45  According  to  Besanko  et  al.  (2000),  a  holdup  problem  arises  when  a  party  in  a  contractual 
arrangement  exploits  the  other  party’s  vulnerability  due  to  relationship-specific  assets). 
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a  continuation  or  renewal  of  the  contract — “frequency”),46  then  efficiency-generating,  specific 
investments  will  not  be  made,  resulting  in  higher  costs.47 

It  is  important  to  note  at  this  point  that  whereas  TCE  has  traditionally  examined  the 
customer-supplier  relationship  in  the  context  of  a  contractual  arrangement,  the  domain  of 
joint  capabilities  acquisition  is  distinctive  (though  not  exclusively  so)  in  the  establishment  of 
partnerships  among  government  entities  (such  as  between  DoD  Services  or  Agencies),  as 
well  as  teaming  arrangements  among  private-sector  enterprises  within  the  defense  industrial 
base  (e.g.,  product  development  contractors  and  their  first-  and  second-tier  suppliers). 

These  internal  relationships  also  incur  transaction  costs  related  to  coordination  and 
motivation  costs,  search  and  information  costs,  decision,  contracting  and  incentive  costs, 
etc.  This  notion  is  explored  further  below. 

Whereas  defense  acquisition  has  traditionally  focused  on  economic  production  costs, 
joint  programs  expose  the  DoD  to  the  potentially  greater  costs  of  managing  the  relationship, 
and  more  importantly,  to  the  risks  of  opportunistic  behavior  on  the  part  of  contracting 
partners — a  critical  “transaction  cost.”  Given  the  multiple  competing  stakeholders  in  a  joint 
SoS  acquisition,  the  principle  of  self-interest  suggests  all  sides  have  incentives  to  behave 
opportunistically  and  may  not  necessarily  have  the  motivation  to  cooperate  to  make  cost¬ 
saving  investments — particularly  when  specific  assets  are  involved,  and  information  is 
imperfect  (incomplete  or  uncertain)  and  asymmetric.48  Clarification  of  the  risks  of 
“opportunism”  (i.e.,  motivation  costs)  is  one  of  the  key  advantages  of  TCE. 

If  it  turns  out  that  joint  Service  SoS  programs  require  a  significantly  greater  ratio  of 
specific  assets  to  total  investments,  then  this  increases  the  risk  of  bilateral  dependency  and 
“hold-up.”  Moreover,  given  the  difficulty  of  writing  complete  contracts  for  joint  SoS  programs 
that  will  cover  every  contingency,  with  incomplete  contracts  the  hold-up  problem  poses 


46  In  terms  of  “frequency,”  past  experience  with  similar  programs  appears  to  have  a 
significant  impact  on  a  supplier’s  costs  and  capabilities.  So,  if  source  selection  and  strategic 
partnership  decisions  recognize  this  and  clearly  favor  past  performance,  the  acquisition 
process  will  be  converted  from  a  one-shot  game  into  a  repeated  game,  allowing  suppliers  to 
earn  a  return  on  their  investment  in  reputation.  In  this  way,  increasing  frequency  through 
strategic  partnerships  and  recurrent  transactions  can  mitigate  opportunistic  behavior  and 
build  trust  in  the  contracting  relationship.  By  identifying  key  characteristics  of  transactions 
such  as  frequency,  and  fully  understanding  their  implications,  decision-makers  could 
mitigate  cost,  schedule  and  performance  breaches. 

47  Scope  for  opportunistic  behavior  may  lead  to  adverse  selection,  choice  of  an  (ex  ante)  inferior 
option  (or  technology),  or  moral  hazard.  Such  scope  increases  risks  that  if  a  relationship-specific 
investment  is  made,  the  other  party  will  exploit  the  terms  of  the  contract  to  “hold  them  up.”  For 
example,  changes  in  specifications  are  frequently  used  by  contractors  as  a  reason  to  raise  prices  and 
profits  under  government  contracts — especially  when  those  investments  by  the  contractor  create 
barriers  to  the  entry  of  competitors. 

48  In  order  to  combat  this  tendency,  and  in  the  spirit  of  resolving  the  principal-agent  problem,  an 
interesting  incentive  clause  is  included  by  the  US  Air  Force  in  its  “National  Polar-Orbiting  Operational 
Environmental  Satellite  System.”  When  establishing  top  executives’  salaries  and  bonuses,  the 
contract  requires  TRW’s  corporate  board  to  consider  contract  performance.  By  tying  senior  executive 
pay  directly  to  contract  performance,  decision-makers  can  help  align  incentives,  increase 
accountability  and  reduce  cost  overruns  (Graham,  2003). 
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additional  risks  for  the  government — such  as  contractors  charging  excessively  high  prices 
for  change  orders,  and  strategic  renegotiations. 


5.  Governance  Issues 

TCE  suggests  the  degree  of  completeness  of  a  contract  is  an  optimizing  decision  by 
the  parties  involved,  one  that  reflects  tradeoffs  between  ex-ante  investments  in  contract 
design  and  the  risk  of  ex-post  costs  of  opportunistic  renegotiation.  In  reality,  contracting 
offers  an  imperfect  solution  to  opportunism.  What  may  be  required  are  additional 
governance  mechanisms  (rules  and  regulations,  reputation  mechanisms,  termination 
agreements,  government-furnished  equipment,  Government-owned,  Contractor-operated 
(GOCO)  facilities,  warranties,  etc.)  to  shift  risks  to  safeguard  and  protect  transaction-specific 
investments,  settle  disputes,  and  adapt  to  new  conditions.  Ex-ante  efforts  may  also  be 
necessary  to  screen  for  reliability  and  reputation  (e.g.,  pre-award  contract  surveys  of 
potential  venders).  These  structures  can  include  anything  from  agreements  to  share  and 
verify  cost  and  performance  information  through  incentive  contracts,  to  the  careful  crafting  of 
dispute-settlement  mechanisms  (e.g.,  alternate  dispute  resolution,  proactive  management 
councils,  etc.).  Among  government  entities  in  joint  acquisition  programs,  memoranda  of 
understanding  or  agreement  (MOUs  or  MOAs)49  reflect  a  “quasi  contractual”  relationship 
among  ostensibly  sovereign  entities.  The  enforceability  of  these  is  always  questionable,  but 
they  nevertheless  serve  to  make  the  particulars  of  agreements  among  the  parties  explicit 
and  provide  both  implicit  and  explicit  dispute-resolution  mechanisms  to  reduce  the  risk  of 
hold-up. 

In  general,  the  less  complex  and  uncertain  a  transaction,  and  the  lower  the 
requirement  for  specific  assets,  the  easier  it  is  to  write  an  explicit  contract  that  covers  all 
contingencies.  Also,  the  lower  the  administrative  and  enforcement  costs  of  that  contract,  the 
lower  the  risk  of  hold-up.  These  favorable  characteristics  are  more  likely  to  exist  in 
established,  traditional  (single  Service),  single  system  acquisition  programs,  and  contribute 
to  lower  costs  (or  cost  overruns),  and  better  performance  and  schedules. 50 


49  The  Air  Force  F-35  Joint  Strike  Fighter  (JSF)  program  featured  elaborate  memoranda  of  agreement 
among  domestic  and  international  partners  to  reduce  the  risk  of  hold-up.  Despite  these  structures, 
the  sovereignty  of  partner  governments  limited  the  enforceability  of  these  instruments,  and  hold-ups 
did  occur — notably  when  the  legislatures  of  the  partner  governments  imposed  changes  to  the 
agreements  articulated  in  the  MOAs,  upon  which  program  plans  were  based.  The  consequential 
impact  of  program  cost,  schedule,  and  performance  outcomes  have  not  been  fully  characterized,  but 
would  be  a  worthwhile  subject  of  future  TCE  research. 

50  If  a  transaction  requires  little  in  the  way  of  specific  assets  (no  hold-up  problem),  and  involves  a 
product  or  service  that  is:  a)  well-defined  and  homogeneous,  b)  easy  to  measure  (limited  complexity 
and  mild  information  asymmetry),  c)  routinely  used  (recurring/frequent  purchases),  d)  not  subject  to 
change  (limited  demand  uncertainty),  and  e)  is  offered  by  competing  suppliers,  then  there  is  little 
room  for  negotiation  (price  and  performance  are  market-driven),  and  the  marginal  benefit  of 
unproductive  bargaining  is  near  zero.  With  little  room  for  bargaining  over  such  routine  and 
uncomplicated  transactions,  substantial  production  and  transaction  cost  stability  can  be  expected  in 
the  acquisition.  Moreover,  since  administrative,  incentive,  and  enforcement  costs  tend  to  be  low  for 
acquisitions  in  more  contestable  (competitive)  markets,  the  marginal  cost  of  engaging  in  the 
transaction  is  relatively  smaller  for  the  military,  and  there  exists  an  incentive  for  the  supplier  to  invest 
in  the  transaction  that  generates  opportunities  for  cost  savings.  International  competition  for  standard 
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Evidence  uncovered  by  Bajari  and  Tadelis  (2001)  in  construction  contracts  reveals 
that  in  cases  in  which  a  transaction  is  easy  to  define  and  measure  (i.e.,  there  is  little 
complexity),  and  only  a  few  minor  changes  are  expected  (i.e.,  there  is  little  uncertainty), 
fixed-price  type  contracts  tend  to  dominate.  However,  the  more  complex  the  transaction — 
the  more  difficult/costly  it  is  to  define  and  measure  performance,  and  the  more  uncertain — 
the  more  likely  it  is  that  a  change  in  the  contract  will  be  required,  and  the  more  severe  the 
adversarial  relationships  experienced  ex-post  when  fixed-price  contracts  are  chosen.  In  the 
latter  case,  fixed-price  type  construction  contracts  often  end  in  costly  renegotiations — in 
which  any  surplus  generated  was  dissipated  in  the  course  of  those  negotiations  through 
unproductive  bargaining  and  influence  activities.  Thus,  complexity  and  uncertainty  can  force 
parties  to  turn  away  from  fixed-price  type  contracts  and  towards  cost-reimbursement  type 
contracts  (e.g.,  costs  plus  a  award/incentive  fee),  and  to  rely  heavily  on  reputation  and  other 
enforcement  mechanisms  to  avoid  ex-post  opportunistic  behavior  that  threatens  to  dissipate 
the  gains  generated  by  a  transaction. 

In  reality,  joint  Service  acquisition  programs  often  involve  highly  interdependent, 
complex  system-of-systems  (SoS)  that  usually  end  up  in  a  bilateral  monopoly  contractual 
setting.51  In  this  case,  assuming  no  specific  assets  are  required,  the  outcome  depends  on 
the  degree  of  contractual  ambiguity  governing  the  transaction,  as  well  as  on  any 
administrative  and  enforcement  costs  involved.  However,  as  complexity,  uncertainty,  and 
opportunism  due  to  specific  investments  increase,  so  does  the  risk  of  hold-up  and  so  do  the 
coordination  and  motivation  (transaction)  costs  required  to  measure,  monitor,  and  govern 
both  the  internal  relationships  among  the  Components  and  the  external  relationship  with  the 
contractor.  These  less-favorable  characteristics  of  joint  SoS  programs  can  discourage 
productive  efforts  and  investments  in  both  internal  Component  relationships  and  external 
contractor  relationships,  and  thus  contribute  to  more  serious  cost  overruns,  schedule 
breaches,  and  performance  shortfalls. 

6.  Coordination  Costs:  Interdependency  yields  Complexity  and 
Uncertainty 

Interdependency  is  typically  defined  as  the  degree  to  which  the  performance  of  one 
activity  (or  system)  relies  on  an  external  activity  (or  system)  for  its  success  (Thompson, 
1967).  Under  conditions  in  which  organizations  are  allowed  to  seek  the  most  efficient  path  to 


(off-the-shelf)  commercial  components  of  weapon  systems  might  be  an  example.  By  unbundling 
large,  complex  weapons  systems  into  sub-systems,  decision-makers  might  reveal  opportunities  to 
enjoy  the  benefits  of  lower  transaction  costs  and  greater  competition,  leading  to  lower  production 
costs.  These  favorable  characteristics  generally  lend  themselves  to  more  accurate  cost  estimating. 

51  Many  factors  conspire  to  create  this  bi-lateral  monopoly.  On  the  buyer  side,  monopsony  power 
partly  derives  from  the  fact  the  military  value  of  most  systems  depends  solely  on  their  performance 
relative  to  the  systems  of  adversaries.  This  is  specific  to  a  country  and  the  defense  environment  it 
faces  at  a  particular  point  in  time,  effectively  making  it  the  sole  buyer  of  a  highly  differentiated 
product.  The  appearance  of  a  superior  alternative  results  in  what  might  be  termed  military 
obsolescence.  Response  to  new  threats  can  require  redesign  during  development,  and  modifications 
during  the  system’s  operational  life.  This  cause-and-effect  relationship  conspires  to  reduce  the 
number  of  buyers  of  a  particular  weapon  system,  since  these  weapon  systems  are  often  evolving 
products  (spiral  acquisition).  Thus,  in  addition  to  technical  uncertainty,  there  is  a  significant  degree  of 
demand  uncertainty. 
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task  accomplishment,  interdependent  relationships  will  be  established  as  long  as  the 
benefits  exceed  the  costs.  Private  entities  typically  make  technology  investments  and  seek 
interdependencies  to  achieve  the  benefits  of  synergy  and  economies  of  scale  based  on 
measurable  effects  on  the  “bottom  line.”  In  contrast,  government  agencies  are  often  guided 
and  constrained  by  legislative  requirements  for  cross-organizational  integration  to  establish 
interdependent  relationships.  Consider,  for  example,  the  increased  emphasis  on  “jointness” 
since  the  Goldwater  Nichols  Department  of  Defense  Reorganization  Act  of  1986.  The  intent 
of  this  legislation  was  clearly  to  increase  interoperability,  and  hence  interdependency,  in  the 
DoD.  Combined  with  recent  reforms  that  reinforce  joint  solutions  to  defense  capability  needs 
(such  as  JCIDS)  and  that  simultaneously  encourage  SoS,  the  result  is  increased 
interdependency  reflected  in  increased  complexity  and  uncertainty,  and  correspondingly 
higher  coordination  costs.52 

For  most  systems,  interoperability  is  pursued  as  a  means  to  leverage  the  collective 
assets  of  various  organizations  located  at  different  points  along  the  value  chain.  For 
example,  in  the  command  and  control  (C2)  process,  military  operations  benefit  when 
commanders  can  seek,  synthesize,  and  disseminate  several  types  of  information  derived 
from  different  organizations.  Experts  in  a  variety  of  areas  must  collaborate  during  the  C2 
process  to  effectively  create  and  execute  battle  plans.  These  experts  may  come  from 
different  disciplines  (or  specialties),  different  branches  of  the  military,  or  even  different 
countries.  In  short,  interoperable  systems  promote  interdependent  actions.  In  turn, 
increased  interdependency  is  reflected  in  increased  complexity  and  uncertainty  and  higher 
coordination  costs. 

Complexity  is  a  key  component  of  transaction  costs.  When  advanced  (immature) 
technologies  are  combined  with  systems  integration  challenges  across  diverse 
organizations  in  the  scale  of  joint  Service  SoS  programs,  the  resulting  complexity  leads  to 
higher  coordination  costs.  Marshall  and  Meckling  (1962)  were  among  the  first  to  discover 
that  variability  in  the  size  of  cost-estimating  errors  in  defense  contracts  could  at  least  partly 


52  Interdependent  activities  are  not  new  to  the  DoD  or  to  government  in  general.  However,  what  is 
new  is  the  scale  in  which  interdependent  actions  are  applied.  Prior  to  the  information  technology 
revolution  of  the  1990s,  spatial  and  temporal  distances  tended  to  impede  communication  and  the 
sharing  of  information  among  partners.  Hence,  tightly  coupled  activities  were  generally  restricted  to 
small  groups  of  geographically  co-located  groups  where  coordination  costs  could  be  minimized.  The 
advent  of  advanced  information  and  communication  technologies  eroded  many  of  the  spatial  and 
temporal  barriers  that  once  thwarted  collaboration.  The  potential  benefits  of  information-sharing 
enabled  by  interoperability  were  then  quickly  realized  and  became  a  major  thrust  for  many 
organizations,  the  DoD  included.  Network  externalities  were  increasingly  recognized — the  value  of 
the  network  to  individual  participants  increased  with  the  number  of  participants  connected  to  the 
network.  This  refers  to  “Metcalfe’s  Law,”  which  proposes  that  the  value  of  a  network  varies  as  the 
square  of  the  number  of  users  or  “nodes.”  The  foundation  of  Metcalfe’s  law  is  the  observation  that  in 
a  communications  network  with  n  members,  each  can  make  (n-1 )  connections  with  other 
participants.  If  all  those  connections  are  equally  valuable,  the  total  value  of  the  network  is  proportional 
to  n(n-1),  that  is,  roughly,  n  2.  The  law  was  named  in  1993  by  George  Gilder,  publisher  of  the 
influential  Gilder  Technology  Report.  Like  Moore's  Law,  which  states  that  the  number  of  transistors  on 
a  chip  will  double  every  18  to  20  months,  Metcalfe's  Law  is  a  rough  empirical  description,  not  an 
immutable  physical  law  (information  derived  from  Briscoe,  Odlyzko  &  Tilly,  2006). 
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be  attributed  to  technological  complexity,  with  larger  errors  associated  with  greater 
technological  advances  sought  in  different  systems.53 

Uncertainty  is  another  key  component  of  transaction  costs.  The  reconciliation  of 
competing  requirements  of  different  players  in  joint  programs  can  lead  to  design  changes 
and  implementation  challenges  (demand  uncertainty).  Similarly,  the  “free  rider  problem”  (in 
which  none  of  the  players  want  to  sacrifice  their  budgets  to  cover  costs  that  might  benefit 
others),  combined  with  changes  in  Congressional  priorities  (political  uncertainty),  can  lead  to 
funding  instability  (budget  uncertainty).  Besides  demand,  political,  and  budget  uncertainty, 
joint  programs  face  measurement  uncertainty,  technological  uncertainty,  supplier 
performance  uncertainty,  etc.54 

An  interesting  avenue  for  future  research  would  be  to  investigate  if  increased 
emphasis  on  “jointness”  since  the  Goldwater  Nichols  Department  of  Defense  Reorganization 
Act  of  1986  or  the  advent  of  the  JCIDS  in  2003  has  increased  the  complexity  and 
uncertainty  of  joint  programs  relative  to  others  and,  consequently,  raised  coordination  and 
motivation  costs.  The  possibility  exists  that  other  characteristics  inherent  in  many  joint 
programs  might  offset  these  higher  transaction  costs. 

On  the  one  hand,  a  joint  program  manager  that  manages  a  program  that  is 
technologically  mature,  that  does  not  require  strict  military  specifications,  in  which  funding 
and  requirements  are  relatively  stable,  and  where  a  contestable  market  exists  for  the 
product  or  service,  may  in  fact  experience  lower  transaction  costs.  On  the  other  hand,  if  the 
program  is  facing  immature  technologies,  rigid  specifications,  funding  and  requirements 
instability,  and  monopolistic  suppliers,  joint  program  managers  and  other  key  decision¬ 
makers  should  recognize  the  potential  for  high  transaction  costs  and  opportunistic  behavior. 


53  Adler  (1995)  examines  the  complex  and  interdependent  relationship  between  product  design  and 
manufacturing,  describing  four  possible  governance  mechanisms  to  improve  coordination  (standards, 
schedules,  mutual  adaptation  and  teams).  McNaugher  (1989)  provides  evidence  that  costs  rise 
rapidly  with  system  complexity,  as  does  the  variance  of  costs  around  expected  costs  (p.  128). 
Consider  the  increase  in  complexity  in  the  US  Navy’s  new  Littoral  Combat  Ship  (LCS)  class,  which 
has  experienced  a  128%  cost  growth  and  which  was  designed  to  avoid  costs  through  smaller  crews 
(substituting  capital  for  labor).  It  is  unclear  whether  the  substantial  increase  in  transaction  costs  and 
scope  for  opportunism  introduced  with  the  increased  complexity  (capital  investment  in  complex 
onboard  systems)  justifies  the  anticipated  labor-cost  savings. 

54  A  dynamic  programming  model  by  Womer  &  Terasawa  (1989)  finds  that  under  demand  uncertainty, 
a  rational  defense  contractor  must  prepare  for  various  contingencies,  and  will,  for  example,  restrict 
investments  in  specific  assets,  which  drives  costs  higher  than  they  would  be  if  demand  were  certain. 
This  tends  to  increase  information  and  contracting  costs,  and  as  the  authors  demonstrate,  threatens 
investments  in  specific  assets.  The  authors  show  that  the  higher  the  probability  the  contract  will  be 
canceled,  the  less  the  contractor  will  invest  in  capital  equipment  (relation-specific  investments),  which 
results  in  relatively  more  labor-intensive  production,  raising  costs.  Thus,  demand  uncertainty 
increases  contracting  costs  and  also  raises  issues  related  to  asset  specificity.  Under  demand 
uncertainty,  the  rational  contractor  will  restrict  investments  in  specific  assets  (such  as  capital  tooling 
or  specialized  expertise),  unless  they  are  reasonably  sure  to  recover  these  costs  via  overhead.  The 
allowability  of  “Facilities  Capital  Cost  of  Money”  as  an  expense  is  a  mechanism  by  which  the 
Government  reduces  the  risk  to  the  contractor  by  allowing  recovery  of  costs  associated  with  specific 
capital  assets  (buildings,  tooling,  etc.)  through  overhead.  Critics  argue  this  encourages  Defense 
contractors  to  over-capitalize,  which  increases  costs  DoD-wide. 
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7.  Conclusion 


Transaction  Cost  Economics  (TCE)  suggests  investigating  the  specific 
characteristics  of  transactions  that  make  up  joint  Service  programs  (like  SoS)  could  help 
anticipate  (and  perhaps  mitigate)  cost  and  schedule  breaches.  Since  increases  in 
interoperability/interdependence  tend  to  increase  complexity  and  uncertainty,  and 
complexity  and  uncertainty  increase  coordination  and  motivation  costs,  it  is  likely  that  cost 
and  schedule  breaches  partly  depend  on  decisions  regarding  the  extent  of  bundling  or 
unbundling  of  the  many  interdependent  parts  of  joint  systems,55  and  on  the  particular  phase 
of  development  or  production  of  those  weapon  systems. 

Historically,  fixed-price  contracts  are  usually  prescribed  in  later  stages  of  product 
development  when  complexity  and  uncertainty  have  been  resolved,  and  the  contract  is 
complete.  In  contrast,  cost-reimbursement  contracts  are  usually  prescribed  in  earlier  stages 
of  product  development  when  complexity  and  uncertainty  have  not  been  resolved,  and  the 
contract  is  incomplete.56  Today  cost-reimbursement  contracts  are  phased  out,  and  fixed- 
price  contracts  phased  in,  as  complexity  and  uncertainty  issues  are  resolved.  However, 
complexity  and  uncertainty  must  be  characterized  in  the  context  of  TCE  for  this  strategy  to 
succeed.  Note  that  while  these  prescribed  contracts  focus  on  the  characteristics  of 
complexity  and  uncertainty,  often  overlooked  is  the  vital  role  of  asset  specificity — a  key 
component  reflected  in  the  motivation  cost  component  of  transaction  costs. 

The  main  strategy  of  reducing  cost  and  schedule  breaches  is  employed  to  identify 
ways  to  cut  coordination  and  motivation  costs.  Specific  recommendations  include:  a) 
reducing  complexity  by  investing  in  a  more  complete  contract — e.g.,  setting  realistic 
baselines  (entailing  higher  search  and  information  costs)  or  using  more  mature  technologies 
(recently  emergent  in  acquisition  policies);  b)  reducing  uncertainty  through  multi-year 
contracts  (reducing  demand  uncertainty)  or  investing  in  a  more  complete  contract  (reducing 
relationship  uncertainty);  c)  increasing  measurement  (CAIG)  and  monitoring  (GAO)  of 
performance  and  both  production  and  transaction  costs  to  reduce  information  asymmetries 
and  the  associated  risks  of  moral  hazard  and  adverse  selection;  d)  placing  credible 
deterrents  to  bad  behavior  in  place — such  as  penalty  clauses,  warranties  and  bonding;  e) 


55  For  instance,  an  extremely  complex  interoperable  system  is  envisioned  for  the  Coast  Guard’s  new 
“integrated  deepwater  system  program.”  The  system  is  intended  to  include  cutters  and  small  boats,  a 
new  fleet  of  fixed-wing  aircraft,  a  combination  of  new  and  upgraded  helicopters,  and  land-  and  cutter- 
based  unmanned  aerial  vehicles  (UAVs) — all  linked  with  Command,  Control,  Communications  and 
Computers,  Intelligence,  Surveillance  and  Reconnaissance  (C4ISR)  systems  and  supported  by 
integrated  logistics.  According  to  a  recent  Coast  Guard  press  release  (2007,  June  25),  “Deepwater  is 
a  25-year,  $24  billion  effort  that  will  produce  more  than  91  new  cutters;  195  new  aircraft”  and  C4ISR 
equipment.  The  lead  systems  integrator  is  a  joint  venture  between  Lockheed  Martin  and  Northrop 
Grumman  that  has  recently  been  in  the  news  for  major  cost  overruns,  schedule  slippages,  and  quality 
issues — the  latter  involving  several  modified  ships  that  were  determined  un-seaworthy. 

56“Complexity”  and  “technological  uncertainty”  (as  opposed  to  “demand  uncertainty”)  are  usually 
correlated.  Ignorance  about  what  buyers  want  and  what  contractors  can  do  result  in  large  up-front 
search  and  information  or  Research  and  Development  (R&D)  costs.  R&D  is  similar  to  a  real  option  in 
the  sense  that  real  options  models  are  learning  models  (Kogut  &  Kulatilaka,  2001).  The  problem  that 
gives  rise  to  high  transaction  costs  in  the  case  of  complex  weapon  systems  is  that  this  characteristic 
of  the  transaction  leads  to  market  failure  (missing  markets).  From  a  TCE  perspective,  the  classic 
market  failures — natural  monopoly,  negative  externalities,  public  goods,  etc. — have  information 
analogues:  missing  markets,  adverse  selection,  and  moral  hazard. 


ACQUISITION  RESEARCH:  CREATING  SYNERGY  FOR  INFORMED  CHANGE  -  348  - 


using  multi-year  contracts  to  gather  information  and  to  reward  good  reputations  (Kelman, 
1990);  f)  mitigating  opportunistic  behavior  introduced  by  asset  specificity  through  careful  use 
of  incentives,  proper  bundling  (or  task-partitioning)  of  joint  programs,  and  strategic 
investments  in  government-furnished  equipment  or  government-owned  and  contractor- 
operated  assets;  and  finally  g)  increasing  market  contestability  through  investments  in  real 
options  (e.g.,  government-controlled  standby  capacity — credible  threat  of  vertical  integration, 
or  second  sourcing — credible  threat  of  entry).57 

We  believe  any  evaluation  of  joint  Service — and  particularly  SoS — acquisition 
programs  would  benefit  from  an  analysis  of  the  characteristics  of  the  corresponding  bundles 
of  transactions  through  the  lens  of  TCE.  An  inspired  effort  to  collect  and  analyze  data  guided 
by  TCE  (as  described  in  this  paper)  could  help  DoD  decision-makers  to  anticipate  and 
mitigate  cost  and  schedule  breaches  and  to  avoid  future  performance  shortfalls. 
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APPENDIX  I 


The  84  DoD  weapon  system  programs  first  divided  into  “Single  Service”  and  “Joint 
Service”  programs  by  Brown,  Flowe  &  Hamel  (2007b)  were  further  divided  into  either  “single 
system”  or  “system-of-systems”  (SoS)  programs.  (Data  available  upon  request.)  The  SoS 
designator  was  used  as  a  proxy  for  complexity.  Of  course,  SoS  programs  are  not 
necessarily  joint  Service  programs.  For  example,  an  aircraft  carrier  qualifies  as  SoS,  but  is 
managed  by  a  single  service.  Another  example  is  the  F-22  managed  exclusively  by  the  Air 
Force.  Of  course,  “single  systems”  can  also  qualify  as  joint  programs  if  they  are  either 
managed  or  procured  by  more  than  one  service.  The  tests  below  focused  on  four  categories 
of  breaches:  Schedule,  RDT&E,  Program  Acquisition  Unit  Cost  (PAUC),  and  Average 
Procurement  Unit  Cost  (APUC). 

Because  the  subdivided  samples  were  not  normally  distributed,  we  ran  a  Kruskal- 
Wallis  (H)  test  to  determine  if  there  was  a  significant  difference  in  the  mean  ranks  of  the 
groups,  testing  Single  Service  vs.  Joint  Service  and  Single  Systems  vs.  System-of-Systems 
(SoS).  The  H  test  is  particularly  robust  as  it  does  not  make  any  assumptions  about  the 
underlying  distribution  of  the  samples. 
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Table  1.  Kruskal-Wallis  (H)  Test 


Variable 

Single 

System 

System 

of 

Systems 

Single 

Service 

Joint 

Service 

Mean 

Rank 

(n=39) 

Mean 

Rank 

(n=45) 

p-value 

Mean 

Rank 

(n=58) 

Mean 

Rank 

(n=26) 

p-value 

Schedule 

breaches 

34.36 

49.56 

0.0039 

38.56 

51.31 

0.0246 

RDT&E 

breaches 

32.44 

51.22 

0.0002 

39.34 

49.56 

0.0631 

PAUC 

breaches 

39.14 

45.41 

0.2244 

41.70 

44.29 

0.6418 

APUC 

breaches 

39.60 

45.01 

0.2633 

42.09 

43.42 

0.7977 

Variables  and  Definitions 


Variable 

Variable  Definition 

Schedule  Breach 

When  schedule  exceeds  most  recent  APB  schedule  estimate 
by  six  months. 

RDT&E  Breach 

When  the  program’s  research,  development,  test,  and 
evaluation  costs  exceed  15%. 

PAUC  Breach 

When  the  Program  Acquisition  Unit  Cost  exceeds  the  most 
recent  Acquisition  Program  Baseline  threshold  by  15%.  This 
is  a  congressionally  reportable  breach. 

APUC  Breach 

When  the  Average  Procurement  Unit  Cost  exceeds  the  most 
recent  APB  threshold  by  15%.  This  is  a  congressionally 
reportable  breach. 

The  results  reported  in  Table  1  suggest  significantly  greater  schedule  breaches  and 
RDT&E  cost  breaches  in  SoS  than  in  Single  Systems,  with  a  weaker  result  for  Joint  Service 
vs.  Single  Service.  The  null  hypothesis  is  that  there  is  no  significant  difference. 

The  results  overall  are  consistent,  but  somewhat  weaker  considering  the  difference 
in  the  average  number  of  breaches  between  development  and  production  programs  using  a 
Mann-Whitney  (U)  test  (which  also  makes  no  assumptions  about  the  underlying  distribution 
of  the  samples).  We  examined  the  difference  in  mean  ranks  for  schedule  breaches,  cost 
breaches  (including  RDT&E,  procurement,  PAUC  and  APUC)  and  total  breaches.  The 
results  are  shown  in  Table  2. 
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Table  2.  Mann-Whitney  Test 


Variable 

Single  System 

vs. 

System -of -Systems 

Single  Service 

vs. 

Joint  Service 

Development 

Production 

Development 

Production 

n=48 

n=36 

n=48 

n=36 

Schedule  breaches 

0.0104 

0.1558 

0.0624 

0.2126 

Cost  breaches 

0.0414 

0.0225 

0.2603 

0.5703 

Total 

0.0276 

0.0289 

0.1632 

0.4497 

These  findings  should  be  interpreted  with  a  note  of  caution.  For  example,  the  limited 
sample,  the  method  for  categorizing  the  degree  of  interdependence  or  “jointness”  and  SoS, 
and  the  failure  to  include  and  control  for  other  important  factors  may  be  significant.  Though 
preliminary,  these  results  offer  evidence  to  support  further  investigations  on  the  role  of 
jointness  and  SoS  in  program  acquisition. 
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Abstract 

This  report  represents  the  authors’  efforts  to  provide  a  useful,  albeit  partial, 
understanding  of  the  international  defense  marketplace — which  we  view  as  growing 
increasingly  complex.  In  pursuit  of  that  objective,  we  provide  a  brief  overview  of 
international  defense  markets  in  the  context  of  both  changes  in  military  affairs  and  the 
various  defense  industrial  bases. 

In  analyzing  the  defense  market,  we  essay  multiple  analytical  frameworks  (along  the 
lines  of  Essence  of  Decision).  Our  analytical  models  are:  (a)  a  sophisticated  view  of  offsets 
in  a  public  policy  context  with  market  imperfections,  (b)  transaction  cost  economics,  with  our 
unit  of  analysis  being  the  nation-state  instead  of  the  firm,  and  (c)  two  standard  corporate 
strategy  models. 

To  test  the  models’  explanatory  powers,  we  consider  three  ongoing  “cases”:  F-35 
Joint  Strike  Fighter,  the  UK  Defense  Industrial  Strategy,  and  the  Northrop-Grumman-EADS 
KC-30  proposal.  Interestingly,  we  find  all  three  hypotheses  have  some  explanatory  power, 
but  none  of  the  three  is  demonstrably  better  than  the  others  (in  this  small  sample). 

Summary 

This  is  a  summary  of  the  report  cited  above  for  inclusion  in  the  Proceedings  of  the 
Fifth  Naval  Postgraduate  School’s  Annual  Acquisition  Symposium  (2008,  May).  (The  report 
itself  greatly  exceeds  the  length  guidelines  for  the  Proceedings.  The  topics  raised  here  are 
discussed  in  greater  detail  within  the  body  of  that  report.) 

The  motivation  for  this  particular  project  was  our  conviction  that  standard  explanatory 
paradigms  for  understanding  global  defense  trade,  e.g.,  offsets  and  related  subjects,  have 
been  strained — perhaps  to  the  breaking  point.  Accordingly,  we  hypothesized  that  analysis 
of  this  increasingly  complex  market  system  would  be  improved  by  the  application  of  multiple 
models  (as  was  done  by  Graham  Allison  in  Essence  of  Decision). 

Section  I  is  a  brief  introduction. 

Section  II  is  an  interpretative  discussion  of  ongoing  developments  in  the  international 
defense  market  place.  In  this  section,  we  discuss  the  following. 
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A.  Military  Affairs.  There  are  two  ongoing  Revolutions  in  Military  Affairs — one  lead  by 
the  US  DoD,  the  other  by  contemporary  terrorist  and  insurgency  movements. 
Accordingly,  the  international  defense  market  has  been  significantly  affected  by  the 
ongoing  competition  between  between  two  groups  that  are  deliberately  engaged  in 
processes  of  rapid  military  innovation  (both  underpinned  by  contemporary 
information  technology). 

B.  Developments  in  the  Global  Defense  Marketplace  (in  general) — including  acquisition 
reform  and  the  globalization  of  defense  industrial  production.  We  noted  in  particular 
the  systemic  tension  between  globalization  and  national  sovereignty — an  issue  that’s 
especially  acute  for  defense-related  goods. 

C.  US  and  EU  Defense  Industrial  Developments  (in  particular).  These  include  the  role 
of  US  and  European  defense  firms  in  the  world  market,  recent  trends  in  defense 
budgets,  and  patterns  of  reorganization  (generally  consolidation).  We  noted  the 
dangers  of  relying  solely  on  detailed  quantitative  analysis  to  analyze  a  complex  and 
changing  system  in  the  absence  of  well-defined  analytic  paradigms. 

Section  III  presents  multiple  analytical  frameworks  as  a  basis  for  the  analysis  of 
complex  systems.  We  then  present  our  three  paradigms. 

A.  We  first  summarize  Allison’s  use  of  multiple  analytical  frameworks. 

B.  Offsets  and  International  Industrial  Participation.  We  hypothesize  that  although 
the  offsets  perspective  was  not  able  to  offer  comprehensive  understanding  of  the 
international  defense  market,  we  believe  it  still  has  explanatory  power  (and  are, 
in  any  case,  obliged  to  assess  its  usefulness). 

C.  Transaction  Cost  Economics  (TCE):  TCE  was  originally  developed  to  study 
vertical  boundaries  of  firms  (the  make-or-buy  decision  in  particular).  We 
summarize  the  relevant  considerations  for  that  decision.  We  expand  the 
standard  TCE  model  somewhat  and  consider  the  nation-state  as  a  military 
enterprise.  (One  would  then  view  a  decision  to  buy  military  equipment  as  a 
decision  to  import — as  opposed  to  a  decision  to  rely  on  domestic  sources.) 

D.  Corporate  Strategy.  Finally,  we  introduce  two  standard  models  of  corporate 
strategy:  Five  Forces  (Porter,  1980)  and  “Co-opetition”  (Brandenburger  & 
Nalebuff,  1996). 

Section  IV  provides  narratives  of  three  ongoing  “cases”:  the  Joint  Strike  Fighter 
(JSF),  the  UK  Defense  Industrial  Strategy,  and  the  KC-30  Proposal.  For  all  three  cases,  we 
provide  an  interpretive  narrative  (up  to  report  publication)  and  then  provide  explanations 
based  on  the  three  frameworks  (from  Section  III). 

Joint  Strike  Fighter 

Offsets:  The  JSF  seems  structured  to  operate  in  an  offsets-free  model.  Among  other 
things,  the  JSF  consortium  is  intended  to  change  foreign  military  sales  customers  to 
stakeholders  (who  share  both  in  risks  and  rewards).  This  arrangement,  in  turn,  is 
intended  to  drive  the  consortium’s  focus  toward  productive  efficiency  rather  than  the 
side  payments  negotiated  in  offset  agreements.  The  viewpoints  of  the  JSF 
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stakeholders  (expressed  in  anonymous  interviews)  is  not  fully  consistent  with  that 
objective. 

Transaction  Cost  Economics:  From  the  prime  contracting  firm’s  perspective, 
international  trade  in  defense  goods  frequently  involves  the  formation  of  long-term 
relations — the  consequences  of  which  are  one  of  the  main  concerns  of  the  TCE 
literature.  The  JSF  strategy  is,  in  a  very  real  sense,  intended  to  delay  the 
“fundamental  transformation”  from  competitive  market  to  something  similar  to 
bilateral  monopoly. 

Corporate  Strategy:  The  JSF  model  addresses  every  defense  industrial  firm’s 
concern  with  buyer  power — sovereign  entities  in  this  case.  The  aim  is  to  mitigate  this 
threat  to  profits  by  recruiting  stakeholders  who  are  well  placed  to  influence  the 
sovereign  buyers’  behavior — in  this  case,  domestic  defense  firms  in  a  number  of 
countries. 

UK  Defense  Industrial  Strategy  (DIS) 

Offsets:  Taken  at  face  value,  the  DIS  is  intended  for  a  post-offsets  trading  regime. 
Flowever,  a  more  in-depth  reading  reveals  a  careful  preparation  for  negotiations  over 
industrial  participation  (as  tacit  offsets).  For  example,  the  DIS  identifies  core 
industries  in  which  there  must  be  domestic  defense  participation. 

Transaction  Cost  Economics:  One  of  the  major  purposes  of  the  DIS  is  protection 
from  the  costs  and  risks  associated  with  outsourcing  (importing)  major  portions  of  the 
UK’s  defense  equipment.  From  this  perspective,  there  are  many  serious  areas  of 
concern  associated  with  importing  modern  defense  systems.  The  DIS’  insistence 
upon  “appropriate  sovereignty”  in  the  lifecycle  management  (and  operational  use)  of 
this  imported  equipment  is  a  serious  effort  to  manage  and  mitigate  those  risks. 

Corporate  Strategy:  The  contemporary  defense  marketplace  poses  significant 
threats  to  the  profits  (and,  therefore,  viability)  of  the  British  defense  industrial  base. 
The  DIS  first  reserves  certain  categories  of  defense  products  to  domestic  firms.  It 
also  insists  upon  the  ability  to  upgrade,  modify  and  generally  manage  defense 
systems  throughout  their  operational  life.  This  is  well  understood  as  a  strategy  for 
mitigating  (foreign)  supplier  power. 

KC-30  Proposal 

Stealthy  Offsets:  The  details  of  the  KC-30  proposal,  which  included  provisions  for 
extensive  US  industrial  participation,  were  really  intended  to  insert  offsets  into  a 
proposal  to  a  customer  who  ostensibly  did  not  engage  in  offset  agreements.  The 
evolution  of  the  KC-30’s  industrial  participation  component  is  best  understood  as  a 
tacit  negotiation  over  offsets. 

Corporate  Strategy:  Standard  models  of  corporate  strategy  clearly  reveal  the 
rationale  for  EADS’  entry  to  the  US  defense  market — with  a  view  to  changing  its 
environment  by  redefining  its  market  niche.  The  USAF’s  KC-X  project  provided 
EADS  with  a  high  degree  of  market  power  relative  to  both  its  rival  and  its  prospective 
buyer.  The  buyer  (the  US  government)  insisted  upon  competition,  with  EADS  being 
the  only  reasonable  competitor  other  than  Boeing.  Hence,  the  success  of  the  KC-30 
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in  the  initial  Air  Force  source-selection  process  is  the  result  of  a  very  well-crafted 
corporate  strategy  (regardless  of  final  outcome). 

Transaction  Cost  Economics:  The  risks  the  US  would  assume  in  choosing  the  KC-30 
include  an  international  politics  of  a  “holdup” — denying  support  to  KC-30s  in  US 
military  operations  because  of  source-country  disapproval.  Hence,  the  partnership 
with  Northrop-Grumman  and  extensive  KC-30  production  work  located  in  the  US  are 
readily  understandable  as  means  to  assure  the  US  government  that  such  risks  are 
not  serious. 

Some  Conclusions 

Basically,  we  found  support  for  our  hypothesis  about  the  international  defense 
market  being  an  increasingly  complex  system,  the  study  of  which  usefully  involves  the 
application  of  multiple  analytical  frameworks. 

We  conclude,  first,  that  all  three  of  our  perspectives  have  explanatory  power  in  all 
three  cases  considered.  Second,  and  also  interesting,  is  that  relative  explanatory  power 
varied  over  this  small  sample.  We  concluded  that  the  offsets  paradigm  was  best  for 
understanding  the  Joint  Strike  Fighter  project;  that  TCE  did  best  for  the  UK’s  DIS;  and 
corporate  strategy  models  provided  especially  good  insights  into  the  KC-30  proposal. 
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Abstract 

Defence  procurement  agencies  in  smaller  countries,  such  as  Australia,  face  a  difficult 
challenge  when  seeking  to  acquire  new  weapons  systems  that  are  either  intrinsically 
complex  or  idiosyncratically  tailored  to  local  needs.  If  they  seek  to  rely  on  production  in¬ 
country,  they  must  draw  on  domestic  infrastructure  almost  certainly  too  limited  in  its 
technological  breadth  and  depth  for  the  task  of  efficiently  producing  a  wide  range  of 
sophisticated  military  products,  which  are  likely  to  be  internationally  uncompetitive,  into  the 
bargain.  If  they  “leave  it  to  the  market”  and  award  contracts  on  the  basis  of  a  competitive 
process  and  set  up  arms-length  relationships  to  pursue  compliance,  they  may  find 
themselves  hostage  to  suppliers  that  cannot  be  replaced  once  the  contract  is  under  way.  If, 
instead,  they  seek  supplies  from  government-owned  arsenals  and  shipyards,  they  may  find 
it  difficult  to  resit  political  pressures  to  preserve  legacy  sectors,  facilities  and  products.  And, 
if  they  look  to  overseas  suppliers  to  meet  their  requirements,  they  usually  lack  the  market 
power  to  negotiate  favourable  terms  and  sometimes  to  have  their  requirements  met  at  all. 

These  challenges,  and  the  related  issue  of  whether  to  use  defence  acquisitions  to 
support  domestic  industry,  are  discussed  in  the  context  of  procurement  of  naval  vessels  and 
support  services  in  Australia.  The  country  has  long  had  domestic  warship  building  and 
maintenance  capabilities,  and  its  shipbuilding  sector  “is  the  jewel  in  the  defence  industry 
crown.  Naval  vessels  are  the  only  major  platforms  built  in  Australia,  and  firms  that  build  them 
are  the  highest  profile  and  most  prestigious  element  of  defence  industry”  (ASPI,  2002,  p.  8). 
In  this  paper,  we  consider  Australia’s  three  post-WWII  shipbuilding  cycles  to  illustrate  how 
successive  federal  governments  have  grappled  with  these  perennial  procurement 
challenges.  We  draw  lessons  from  the  first  two  cycles  to  explain  why  and  how  the 
government  has  been  driven  to  increase  its  responsibility  for  managing  key  shipbuilding 
projects  following  a  period  of  seeking  to  rely  more  heavily  on  the  forces  of  competitive 
industrial  supply  and  after  nearly  a  decade  of  “smart  procurement”  rhetoric. 

Many  lessons  apparent  in  the  context  of  Australian  naval  shipbuilding  and  repair  are 
relevant  to  other  defence  industry  sectors.  They  illustrate  a  range  of  problems  typical  of 
many  Defence-Industry  relationships  in  small  countries:  local  production  vs.  imports,  political 
and  economic  vs.  strategic  aspects  of  in-country  sourcing  of  materiel,  the  role  of 
competition,  the  management  of  procurement  risks,  and  business  models  used  to  engage 
suppliers  and  mitigate  the  risks  of  inadequate  supplier  performance. 
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Introduction 

Naval  shipbuilding  has  consistently  captured  the  attention  of  the  Australian  public 
more  than  any  other  area  of  defence  procurement — from  the  troubled  government  shipyards 
of  the  1950s  and  1960s  through  to  the  Collins  submarine  project  of  the  1990s  (ASPI,  2002). 
This  is  partly  because  naval  vessels  are  the  only  major  platforms  built  in  Australia.  And, 
partly,  because  the  procurement  of  naval  assets  and  the  disposition  of  industry  facilities 
have  long  been  highly  politicised,  with  the  federal  government  balancing  the  competing 
interests  of  different  states,  services  and  industries.  As  a  result,  the  Australian  naval 
shipbuilding  and  repair  sector  has  consistently  presented  governments  of  the  day  “with  a 
series  of  interwoven  (procurement)  challenges”  (p.  1 ).  The  response  to  such  challenges  has 
been  embedded  for  20  years  or  more  in  economic  and  institutional  reforms  applying  to 
Australian  defence  industry  overall,  and  the  sector  has  benefited  particularly  from  reforms  to 
the  defence  supply  chain.  However,  while  considerable  improvements  have  been  achieved, 
accounts  of  project  delays  and  cost  overruns  still  attract  media  attention  and  embarrass 
successive  governments. 

By  the  standards  of  older  maritime  nations,  the  industry  may  not  have  a  long  history 
but  it  has  had  its  share  of  both  failure  and  success.  Within  the  former,  arguably  the  most 
publicised  has  been  the  procurement  of  the  Collins  Class  submarines  in  the  1980s  and 
1990s.  No  other  Australian  defence  acquisition  in  recent  years  has  had  a  more  profound 
impact  on  how  the  government  goes  about  the  acquisition  of  major  strategic  capabilities  and 
on  the  mechanics  of  the  defence  acquisition  process.  As  the  latest  cycle  of  naval 
acquisitions  has  started  to  unfold,  the  key  question  is  what  lessons  have  been  learned  and 
with  what  implications  for  new  process. 

This  would  become  clear  as  the  Australian  government  seeks  to  handle  two  forms  of 
competition.  On  the  one  hand,  prime  contractors,  main  systems  integrators  and  some 
original  equipment  manufacturers  (OEMs)  compete  for  a  market — the  market  in  new 
capability  elements — by  offering  alternative  specifications  of  weapons  systems  sought  by 
Defence  (i.e.,  different  platform  designs  and  combat  systems).  On  the  other  hand,  lower-tier 
contractors  compete  in  a  market,  the  market  for  work  subcontracted  by  primes  and  higher- 
tier  contractors.  In  each  case,  local  content  requirements  and  defence  procurement  policy 
have  a  role  to  play. 

This  paper  is  structured  as  follows.  First,  we  briefly  review  the  history  of  naval 
shipbuilding  and  repair  in  Australia.  We  discuss  the  three  post-World  War  II  ship  acquisition 
cycles,  which  are  critical  to  the  understanding  of  how  successive  Australian  governments 
have  approached  naval  acquisitions  and  defence  procurement  in  general.  Second,  we 
consider  the  in-country  maritime  industry  (i.e.,  shipbuilders  and  maintainers,  facilities  and 
industry  disposition,  the  changing  division  of  labour  and  the  associated  supply  chains). 
Finally,  we  draw  some  lessons  from  the  Australian  naval  shipbuilding  experience. 

Naval  Shipbuilding  in  Australia 

Post-war  shipbuilding  cycles 

During  World  War  II,  Australia  became  a  significant  builder  and  repairer  of  naval 
vessels.  In  total,  113  naval  ships  were  built  for  the  Royal  Australian  Navy  (RAN)  at  ten 
Australian  dockyards.  Also,  4,000  ship  repairs  were  completed  for  the  RAN,  500  for  the  US 
Navy  (USN)  and  nearly  400  for  the  Royal  Navy  (Parliament  of  Australia,  2006,  p.  41 ).  The 
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scale  of  activity  declined  significantly  during  the  post-war  period.  Since  1945,  Australia  has 
experienced  two  major  naval  building  cycles  separated  by  a  15-year  period  of  low  activity 
and  a  third  was  under  way  by  2008  (ASPI,  2002). 

In  the  1950s  and  1960s,  the  first  cycle  saw  nine  destroyers  built  at  the  government- 
owned  dockyards.  The  cycle  also  included  an  afloat  support  ship,  hydro  vessels  and  patrol 
boats.  In  the  1970s  and  early  1980s,  no  surface  combatants  were  built  in  Australia; 
although,  four  large  naval  support  and  hydrographic  vessels  were  completed,  as  well  as 
eight  heavy  landing  craft  (LCH)  craft  and  14  patrol  boats. 

The  second  naval  combatant  building  cycle  began  in  1984  with  an  order  for  two 
guided  missile  frigates  (FFGs).  This  was  followed  by  orders  for  six  Collins  Class 
submarines,  ten  ANZAC  Class  frigates  (involving  workshare  arrangements  with  New 
Zealand),  six  minehunters  and  other  ships  (e.g.,  patrol  boats  and  two  hydrographic  ships). 
The  second  cycle  will  end  with  the  commissioning  into  service  of  the  last  Armidale  Class 
patrol  boats  in  the  early  2000s. 

The  third  cycle  (much  smaller  by  volume)  began  in  the  late  2000s,  with  tenders  for 
the  construction  of  three  air  warfare  destroyers  (AWD),  two  large  landing  helicopter  dock 
(LHD)  ships,  afloat  support  ships  and  the  watercraft  element  of  the  amphibious  deployment 
and  sustainment  (ADAS)  project.  This  cycle  of  naval  shipbuilding  is  expected  to  end  around 
2016-17,  with  the  next  cycle  expected  to  start  around  2018. 

The  first  cycle:  The  troubled  years 

The  construction  of  destroyers  in  the  1950s  and  1960s  was  notorious  for  its  cost 
overruns,  schedule  slippages  and  industrial  disputes.  As  noted  by  the  Australian  Senate 
inquiry  into  naval  shipbuilding,  “Australia’s  increasing  resort  over  the  1960s  and  1970s  to 
purchasing  foreign  naval  vessels  for  the  RAN  reflected  the  poor  performance  of  domestic 
naval  shipbuilding  projects”  (Parliament  of  Australia,  2006.  p.  41).  Apart  from  two 
oceanographic  ships,  the  government-owned  dockyard  at  Williamstown  in  Melbourne  did  not 
commission  a  naval  vessel  between  1971  and  1991  (p.  42).  And,  after  launching  HMAS 
Torrens  in  1968,  the  other  major  government-owned  dockyard  at  Cockatoo  Island  in  Sydney 
did  not  commission  another  naval  vessel  until  1986.  Thus,  no  warships  were  launched  in 
Australia  for  over  20  years.  This  preference  for  imports  left  Australian  naval  shipyards  with 
mostly  repair  and  (limited)  refit  work  (p.  42). 

An  example  of  local  construction  problems  in  the  1970s  was  the  ill-fated  DDL  (light) 
destroyer  project  approved  in  1972.  Starting  in  1975,  three  locally  designed  ships  were  to  be 
built  at  the  Williamstown  dockyard.  While  it  was  accepted  that  cost  premia  for  local  build 
were  to  be  incurred,  they  were  justified  on  the  grounds  that  local  shipyards  would  later  be 
best  positioned  to  provide  logistic  through-life  support  and  battle  damage  repairs.  Also, 
investments  needed  to  pump-prime  local  shipbuilding  capabilities  were  aimed  at  enhancing 
the  in-country  skill  base  and  technological  know-how.  The  project  was  cancelled  in  1973  as 
the  Navy  and  the  Department  of  Defence  found  the  initial  cost  estimates  to  be  grossly  over- 
optimistic  and  a  Joint  Parliamentary  Committee  took  the  position  that  risks  inherent  in  a  local 
design  were  excessive  (p.  43).  A  lesson  drawn  from  this  experience  was  “the  need  for 
tighter  controls  on  Navy’s  design  requirements.  Part  of  the  problem  was  that  those  involved 
with  the  specifications  for  the  project  were  without  responsibility  for  cost  and  schedule”  (p. 


43). 
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Following  the  cancellation  of  the  DDL  project,  the  government  turned  to  overseas 
shipyards  to  initiate  the  acquisition  of  guided  missile  frigates.  In  1974,  the  purchase  of  two 
imported  FFGs  was  approved  by  the  government.  The  vessels  were  to  be  built  in  the  USA 
and  acquired  under  Foreign  Military  Sales  (FMS)  arrangements  managed  by  USN.  The 
purchase  of  a  third  FFG  was  approved  in  1977  and  a  fourth  in  1980.  The  ships  were 
delivered  between  1980  and  1984,  mostly  on  schedule,  but  the  cost  of  acquisition  ballooned 
(Parliament  of  Australia,  2006,  pp.  43-44).  In  part,  the  cost  overrun  was  due  to  higher-than- 
anticipated  inflation  and  exchange  rate  re-alignment.  But  the  purchase  also  revealed 
systemic  problems  in  Australian  defence  procurement.  As  new  technologies  emerged,  the 
first  three  frigates  had  to  be  retrofitted  with  long-range  sonar  systems  and  more  capable 
helicopters.  The  design  of  the  fourth  FFG  was  altered  to  incorporate  several  modifications 
requested  by  the  RAN.  It  was  argued  that  some  cost  overruns  might  have  been  avoided 
“had  the  RAN  seized  opportunities  to  incorporate  modifications  during  the  construction 
phase”  (p.  44).  The  procurement  process  was  thus,  apparently,  neither  sufficiently  flexible 
nor  agile  to  cope  with  changes  in  the  technical  specification  of  the  deliverables. 

There  were  also  problems  with  the  use  of  FMS  arrangements.  A  1974  Memorandum 
of  Agreement  with  the  US  allowed  Australia  to  withdraw  from  the  project  if  the  ships  failed  to 
meet  the  RAN’s  requirements  or  turned  out  to  be  “unacceptably  costly.”  However,  the  USN 
and  the  US  Department  of  Defense  resisted  changes  requested  by  Australia  (p.  44).  The 
Agreement  also  included  a  limited  “offset-type”  provision  to  make  Australian  industry 
manufacture  and  supply  components  for  the  RAN  and  the  USN  FFGs.  The  actual  FMS 
arrangement,  as  opposed  to  what  was  initially  envisaged,  frustrated  all  such  local  content 
initiatives  and  restricted  the  scope  for  Australian  industry  participation.  A  lesson  drawn  from 
this  experience  was  that  “in  future,  it  was  necessary  to  sign  deeds  of  agreement  with  the 
prime  contractors  before  negotiating  a  Letter  of  Offer  and  Acceptance  with  the  US 
government”  (p.  44).  Another  lesson  drawn  was  that  Australian  industry  was  not  competitive 
enough  to  win  work  on  its  own  merit  and  that  earlier  participation  of  potential  suppliers  was 
needed  at  the  project  planning  stages  if  any  in-project  import  substitution  was  to  be 
achieved  (p.  45). 

By  the  end  of  the  1970s,  it  was  apparent  that  the  naval  shipbuilding  sector  in 
Australia  was  suffering  from  deeply  ingrained  systemic  problems.  Industrial  relations  were 
particularly  bad  as  naval  shipyards  were  seen  by  both  the  unions  and  the  shipyard 
management  as  Defence-funded  sheltered  workshops  (Parliament  of  Australia,  2006).  The 
Department  of  Defence  lacked  the  ability  to  specify  its  needs  precisely  enough  to  prevent 
endemic  requirements  creep.  It  also  lacked  effective  contracting  skills.  This  was  an 
important  limitation  as  changing  technologies,  especially  the  growing  use  of  electronics  and 
information  technology,  made  naval  vessels  increasingly  complex  and  knowledge-intensive. 
Project  management  skills  were  also  lacking  in  defence  and  there  were  shortages  of  critical 
shipbuilding  skills  at  naval  yards. 

Fresh  start 

The  1976  Defence  White  Paper  foreshadowed  aspirations  to  develop  local  defence 
industry  capabilities  to  enhance  Australia’s  defence  self-reliance.  In  line  with  this  aim,  the 
Australian  Frigate  Project  (AFP)  was  initiated  in  1978.  The  FFG-7  Class  frigate  would  be 
constructed  locally  to  an  imported  design  that  was  seen  as  flexible  enough  to  accommodate 
high  local  content  requirements.  In  1980,  the  government  decided  to  build  two  FFG-7 
frigates  at  the  Williamstown  dockyard,  providing  that  the  shipyard  “demonstrated  its  capacity 
to  build  the  ships  to  the  RAN’s  requirements”  (pp.  46-47). 
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The  arrival  of  a  new  Labor  government  in  1983  brought  with  it  the  re-affirmation  of 
the  self-reliance  objectives,  accompanied  by  a  commitment  to  microeconomic  reform  aimed 
at  increasing  the  competitiveness  of  Australian  manufacturing  industry.  The  government 
was  keen  to  build  warships  in-country  providing  that  significant  improvements  in  shipyard 
productivity  could  be  negotiated  and  delivered.  Preferably,  this  was  to  be  achieved  through 
transferring  ownership  to  the  private  sector.  Privatisation  of  government  factories  and 
shipyards,  including  naval  shipbuilding  facilities,  was  seen  by  the  government  as  an 
essential  part  of  its  broader  package  of  “microeconomic”  reform.  The  new  government  was 
also  ready  to  confront  unions  by  resisting  their  demands  to  build  a  tanker  at  Cockatoo  Island 
“ultimately  condemning  the  yard  to  extinction”  (p.  47). 

The  two  FFGs  were  to  be  built  at  the  Williamstown  dockyard  providing  that  its 
productivity  could  be  lifted,  cost  and  schedule  discipline  imposed,  and  a  series  of 
enforceable  agreements  concluded  to  tighten  work  practices  and  restrict  the  drift  of  product 
specifications.  A  contract  between  the  Department  of  Defence  (customer)  and  the 
Department  of  Defence  Support  (contractor)  was  signed  in  1983  for  the  two  ships  to  be 
delivered  between  1992  and  1994.  The  contract  was  to  facilitate  extensive  local  industry 
involvement  to  enhance  national  defence  self-reliance  and  navy  preparedness.  As  revealed 
by  the  1986  review  of  the  project  by  the  Joint  Committee  of  Public  Accounts,  the  project 
budget  included  a  cost  premium  for  the  local  build  of  about  30%  (p.  48).  In  1987,  the 
government  sold  the  Williamstown  naval  dockyard,  with  the  FFG  arrangement,  to  the 
Australian  Marine  Engineering  Corporation  (AMEC).  The  privatisation  of  the  yard  turned  out 
to  be  a  very  successful  initiative.  Both  ships  were  launched  by  AMEC  ahead  of  their  initially 
agreed  schedule  and  within  the  original  cost  estimates  (in  real  terms).  The  only  real  cost 
increase  was  attributed  to  the  privatisation  process  perse.  Further,  local  industry  content 
accounted  for  90%  of  the  AMEC-borne  cost  and  75%  of  the  total  project  cost  (Parliament  of 
Australia,  2006). 

The  second  cycle:  Back  on  track 

While  the  beginning  of  the  second  shipbuilding  cycle  may  be  associated  with  the 
FFG  project,  it  really  unfolded  in  the  late-1980s.  The  1987  Defence  White  Paper  reaffirmed 
the  Labor  government’s  commitment  to  the  development  of  competitive  local  defence 
industry  capabilities,  particularly  in  the  shipbuilding  sector,  and  to  the  policy  of  defence  self- 
reliance.  The  second  cycle  got  under  way  following  the  government’s  decision  to  build  six 
Collins  Class  submarines,  awarding  the  contract  to  the  Australian  Submarine  Corporation  in 
1987;  and  ten  ANZAC  (Meko  200  Class)  frigates,  the  contract  going  to  the  AMEC- 
Blohm+Voss  consortium  in  1989.  In  1994,  another  major  contract  was  awarded  to  the  then 
Australian  Defence  Industries  (ADI) — now  Thales — this  time  to  build,  to  an  Italian  design,  six 
Huon  Class  coastal  minehunters.  The  final  contract  in  the  second  cycle  was  signed  in  2003 
with  Defence  Maritime  Services  Pty  (DMS),  a  joint  venture  between  P&O  Maritime  Services 
and  Serco  Australia  for  the  delivery  of  14  Armidale  Class  patrol  boats.  The  fleet  was  built  by 
Austal  Ships  Ltd.,  Australia’s  largest  commercial  shipbuilder,  and  is  to  be  fully  supported  by 
DMS  throughout  its  service  life  (Kerr,  2008a).  The  second  cycle  also  included  some  minor 
naval  construction  (e.g.,  Freedom  Class  patrol  boats  and  hydrographic  ships).  In  contrast  to 
previous  periods,  nearly  all  ships  required  by  the  RAN  during  the  second  cycle  were  built  in¬ 
country.  The  main  projects  of  the  second  cycle  are  discussed  below. 
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Collins  Class  submarine  project 


To  facilitate  building  submarines  in  Australia,  the  Australian  Submarine  Corporation 
(ASC)  was  established  in  1985  as  a  joint  venture  between  Sweden’s  Kockums  (as 
shipbuilder  and  designer  holding  49%  of  the  company’s  shares),  the  Australian  government- 
owned  Australian  Industry  Development  Corporation  (49%)  and  Wormalds  International  and 
Chicago  Bridge  and  Iron  (holding  the  2%  balance  of  shares).  In  1987,  ASC  was  chosen  as 
the  prime  contractor  for  the  fixed  cost  Aus$3.9  billion  (1986  prices)  project  to  deliver  six 
submarines.  With  over  73%  local  content  for  the  six  platforms,  at  least  3,500  suppliers,  and 
1,600  individual  contracts  (Parliament  of  Australia, 2006,  p.  56),  the  project  was  “Australia’s 
most  ambitious  and  technically  advanced  defence  project  ever”  (McIntosh  &  Prescott,  1999, 
p.  5). 

The  submarines  experienced  much  publicised  teething  problems  but  were  eventually 
acclaimed  as  “world  class”  (Parliament  of  Australia,  op.  cit.).  The  main  early  problem  was 
attributed  to  a  decision  to  acquire  a  sophisticated  combat  data  system  (CDS)  independently 
of  the  platform  design  when  the  most  straightforward  approach  would  have  been  to  select  a 
design  with  the  CDS  fitted  as  standard  (Woolner,  2006,  p.  72).  This  was  compounded  by 
Navy’s  preference  for  the  CDS  to  be  developed  in  order  to  meet  its  unique  requirements 
rather  than  purchased  as  a  military-off-the-shelf  (MOTS)  system  (Woolner,  2001,  p.  9).  “By 
including  the  combat  system  with  the  platform  in  the  single  prime  contract,  with  a  unique 
military  specification,  Defence  left  itself  widen  open  to  [...]  technological  problems” 

(McIntosh  &  Prescott,  1999).  By  1993,  it  had  become  apparent  that  Rockwell,  the  CDS  sub¬ 
contractor  and  designer,  was  not  able  to  meet  Navy’s  specifications.  Nevertheless,  Defence 
did  not  authorise  a  replacement  MOTS  system.  The  first  submarine  was  provisionally 
accepted  into  service  in  1996  with  the  combat  system  incomplete  and,  by  the  late  1990s,  the 
Collins  Class  project  had  become  a  major  embarrassment  for  Defence  and  the  government. 

In  1999,  the  government  terminated  the  failed  CDS  sub-contract  and  sought  another 
CDS  contractor  through  open  competition.  In  2001 ,  however,  the  government  decided  to 
scrap  the  tender  process  and  awarded  the  contract  to  the  US  firm  Raytheon.  Later  that  year, 
the  RAN  and  the  USN  signed  an  agreement  to  cooperate  in  equipment-sourcing  and  logistic 
support  and  to  enhance  Collins  Class  interoperability  with  US  ships.  The  German  STN  Atlas 
was  also  awarded  a  contract  for  sonar  and  navigation  equipment  (Parliament  of  Australia, 
2006,  pp.  59-60). 

The  sequence  in  which  the  six  hulls  were  constructed  allowed  for  little  learning  by 
doing.  As  a  former  high-ranking  naval  officer  argued  during  a  2006  parliamentary  inquiry, 
“there  is  a  need  to  have  an  increased  gap  between  the  lead  ship  of  a  class  and  its 
successor.  The  lead  ship  needs  to  be  evaluated  and  give  the  all  clear  before  the  successor 
is  completed”  (Parliament  of  Australia,  2006,  footnote  31,  p  59).  Instead,  the  ships  were 
largely  batch-manufactured  and  batch-constructed.  While  economies  of  scale  and  scope  are 
unavoidably  lost  through  fragmenting  the  sequence  of  ship  construction,  there  is  more 
opportunity  to  alter  the  specifications  of  successor  ships  by  learning  from  the  in-service 
performance  of  their  predecessors.  This  principle  of  “spiral”  or  incremental  new  capability 
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formation  was  well  understood  and  practiced  in  Sweden  where  the  Collins  Class  design 
originated.58 

Criticism  has  also  been  directed  towards  how  the  project  was  commissioned  and 
managed  by  Defence.  A  fixed-price  contract  was  used  to  avoid  cost  overruns  associated 
with  traditional  cost-plus  contracts,  and  to  shift  most  product-  and  (construction)  process- 
related  risks  from  the  Commonwealth  to  the  contractor.  However,  the  use  of  a  fixed-price 
contract  for  that  reason  was  flawed  on  three  accounts: 

■  for  a  country  lacking  experience  as  the  builder  of  modern,  sophisticated  weapons 
systems,  the  magnitude  of  the  technological  challenge  inherent  in  this  project  was 
grossly  underestimated  by  both  by  the  ASC  as  a  contractor  and  by  Defence  as  a 
customer.  There  was  too  much  reliance  on  Kockums’  expertise  as  a  builder  of 
submarine  platforms  and  a  rather  poor  understanding  of  technological  challenges 
posed  by  the  development  of  the  bespoke  CDS.  In  such  circumstances,  the 
Commonwealth  (Defence)  might  have  realised  the  limitations  of  risk-shifting  between 
the  parties  and,  instead,  relied  on  risk-sharing  mechanisms  such  as  those  provided 
by  incentive  contracts  and  risk  mitigation  through  more  collaborative  management  of 
the  project; 

■  given  the  developmental  nature  of  the  project,  the  use  of  a  fixed  price  contract 
provides  little  effective  protection  for  the  buyer  (Defence)  since  contract  variations 
are  inevitable.  An  ex-ante  fixed-price  contract  may  in  reality,  become  an  ex-post, 
cost-plus  arrangement.  If  contract  variations  are  regularly  approved,  there  is  no 
incentive  for  the  contractor  to  seek  cost  efficiencies.  It  would  have  been  preferable  to 
use  a  flexible  form  of  contract  to  allow  for  learning,  to  provide  incentives  to  improve 
and  share  risks  rather  than  to  end  up  with  the  de  facto  cost-plus  arrangement 
dressed  up  as  a  fixed-price  contract;  and 

■  a  belief  that  project  risk  could  be  shifted  to  the  contractor  to  reduce  the 
Commonwealth’s  exposure  was  naive,  given  the  ASC  equity  structure.  With  its  49% 
share  of  equity,  the  Commonwealth  was  both  the  sole  buyer  of  the  ships  and  a  key 
shareholder  on  the  supply  side.  In  2000,  when  Kockums  was  acquired  by  the 
German  submarine  builder  HDW,  the  Australian  government  stepped  in  to  buy  the 
Kockums’  share  of  ASC  equity. 

This  contractual  debacle  was  summarised  by  an  Australian  parliamentary  researcher 

thus: 

The  most  compelling  lesson  that  can  be  learnt  from  the  Collins  submarine  program  is 
the  importance  of  selecting  the  procurement  strategy  to  suit  the  nature  of  the  project. 
In  hindsight,  the  point  where  it  was  decided  to  develop  a  unique  design  for  the  new 
submarines  was  the  time  to  change  the  procurement  strategy.  (Woolner,  2001,  p.  47) 

The  nationalisation  of  ASC  was  an  embarrassment  for  a  government  overtly 
committed  to  the  privatisation  process:  “There  was  more  than  a  touch  of  irony  in  the  fact  that 


58  Sweden  has  traditionally  ordered  its  submarines  in  very  small  batches  to  allow  for  benefits  of 
learning-by-doing  and  technological  change  to  be  continuously  absorbed  into  subsequent  designs — 
even  though  it  has  been  well  understood  that  cost  premia  would  be  incurred  as  a  result  of  fragmented 
production. 
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after  decades  of  effort  to  transfer  all  defence  production  capability  to  commercial  industry, 
the  Government  finds  itself  the  owner  of  ASC”  (ASPI,  2002,  p.  24).  But,  the  nationalisation 
of  ASC  also  exposed  a  more  serious  flaw  in  the  procurement  philosophy  that  was  inherent 
in  the  Collins  Class  acquisition.  Under  the  original  contract,  Kockums  retained  much  IP  in 
the  vessel’s  design.  The  ASC  shareholding  arrangement  made  it  difficult  to  determine  the 
ownership  of  various  IP  changes  to  the  original  design,  new  IP  elements  and  the  associated 
body  of  design  data  that  were  critical  to  access  if  ASC  was  to  carry  on  as  the  ship’s 
maintainer  and  modifier.  The  resultant  legal  dispute  took  until  2004  to  resolve.  Under  the 
new  arrangement,  Kockums  owns  the  legacy  IP  but  ASC  has  full  access  to  it  (Parliament  of 
Australia,  2006,  p.  55). 

The  introduction  of  sensitive  US  technology  into  the  vessels  and  the  involvement  of 
the  US  firm  Electric  Boat  as  a  capability  partner  with  ASC  added  another  degree  of 
complexity  to  the  IP  dispute.  The  inadequacies  of  the  Collins  Class  technology  management 
highlight  the  critical  importance  of  access  to  proprietary  technological  know-how  and  IP  in  all 
knowledge-intensive  projects.  This  is  often  poorly  understood  in  large,  technologically 
complex,  developmental  projects  in  which  a  detailed  design  does  not  exist  at  the  time  a 
contract  to  proceed  with  the  project  is  signed.  Thus,  a  classic  “hold-up”  relationship  may 
emerge  between  the  parties  as  the  buyer  belatedly  realises  that  its  ownership  of  an  asset  is 
incomplete  without  the  transfer  of  all  IP.  The  incompleteness  of  ownership  rights  imposes 
severe  limitations  on  who  is  allowed  to  maintain  the  asset  and  who  has  the  right  to  modify  it. 
By  the  time  the  buyer  becomes  aware  of  such  problems,  the  cost  of  contract  re-negotiation 
may  be  prohibitive  and  opportunities  for  switching  suppliers  limited.  This  problem  is 
compounded  when  the  product  design  incorporates  “black  boxes,”  which  can  only  be 
accessed  by  the  original  supplier  or  its  agent  and  which  are  subject  to  technology 
restrictions  imposed  by  the  supplier’s  home  government. 

In  sum,  the  Collins  Class  project  “exposed  serious  flaws  in  defence’s  procurement 
processes”  (Parliament  of  Australia,  2006,  p.  57).  Its  well-publicised  difficulties  were  not  only 
embarrassing  for  the  government  but  also  made  the  government  determined  to  change  the 
nature  of  its  principal-agent  relationship  with  Defence.  Following  yet  another  review  of  new 
capability  formation  and  procurement  management  by  Defence  ( Kinnaird  Report),  the 
government  decided  to  restructure  the  Defence  Materiel  Organisation  (DMO — its 
procurement  agency)  into  a  “prescribed  agency”  (partially  detached  from  Defence  and 
reporting  directly  to  the  government)  to  handle  defence  procurement  and  through-life 
capability  support.  In  particular,  DMO  was  to  foster  the  kind  of  professional  project 
management  expertise  required  to  bring  rigour  and  experience  into  the  procurement 
process  and  to  end  the  long  tradition  of  well-intended  and  energetic  but  sometimes 
amateurish  project  management. 

ANZAC  frigate  project 

At  Aus$7  billion  (2006  prices),  the  ANZAC  Frigate  project,  was  the  largest  single 
Defence  design  and  construction  contract  awarded  in  Australia  in  the  closing  decades  of  the 
20th  century.  It  was  also  the  only  European-style  naval  “workshare”  contract.  There  were 
two  customers,  the  navies  of  Australia  (eight  ships)  and  New  Zealand  (two  ships)  and  the 
industry  workload  was  shared  between  the  two  countries.  It  was  expected  that  neither  navy 
would  cross-subsidise  the  shipbuilding  costs  of  the  other;  sub-contractors  were  to  be 
selected  competitively;  and  the  achieved  workshare  between  the  two  countries  was  to 
reflect  the  overall  cost  shares.  The  frigates  were  assembled  at  the  recently  privatised  AMEC 
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shipyard  at  Williamstown  with  modules  built  at  this  and  other  shipyards  in  Australia  and  New 
Zealand.  During  the  contract  life,  the  shipbuilder  changed  its  name  twice  to  finally  become 
Tenix  Marine  Division  of  Tenix  Defence  Pty  Ltd.  (Parliament  of  Australia,  2006).  By  the  early 
2000s,  Tenix  Defence — incorporating  the  marine  division — had  become  one  of  Australia’s 
largest  defence  contractors. 

Despite  its  initially  limited  experience  as  a  shipbuilder,  Tenix  completed  the  project 
on  schedule  and  on  budget.  This  outcome  was  helped  by  the  modular  ship  construction  and 
by  a  collaborative  and  highly  synergistic  arrangement  with  SAAB,  the  combat  system 
supplier,  to  test  the  combat  system  prior  to  installation  (Tasman  Asia  Pacific,  2000,  p  9).  A 
requirement  for  the  project  was  to  achieve  high  levels  of  local  content  (the  then  government 
policy  of  Australian  Industry  Involvement,  All).  This  was  in  part  accomplished  through 
effective  sub-contracting  with  the  help  of  the  Industrial  Supplies  Office  (ISO),  an  agency  set 
up  to  assist  small  and  medium  sized  enterprises  (SME)  in  broadening  their  customer  base. 
The  search  for  subcontractors  to  meet  the  All  target  sometimes  involved  what  a  Tenix 
manager  described59  as  “reverse  garage  sales,”  i.e.,  components  were  put  on  display  and 
SMEs  were  invited  to  decide  which  of  these  products  could  be  made  locally.  This  approach 
to  sub-contracting  has  been  acclaimed  as  a  factor  contributing  to  the  project’s  cost  and 
schedule  discipline  and  copied  by  other  projects  (Tasman  Asia  Pacific,  2000). 

In  2001,  Tenix,  SAAB  and  the  DMO  (Defence)  signed  a  tripartite  long-term  alliance 
agreement  (the  first  of  its  kind)  to  provide  in-service  support  for  the  ANZACs  and  to 
collaborate  in  future  modifications  and  capability  enhancements  of  the  class  (Parliament  of 
Australia,  2006).  This  agreement  concluded  Tenix’  involvement  in  the  second  shipbuilding 
cycle  and  positioned  the  company  favourably  as  a  bidder  for  construction  work  in  the  third 
cycle  (see  below). 

Minehunter  coastal  project 

In  1989,  Australian  Defence  Industries  (ADI)  was  formed  as  a  corporatised, 
government-owned  entity  set  up  to  consolidate  major  defence  industry  facilities  still  in 
government  ownership.  This  included  naval  engineering  at  the  Garden  Island  dockyard  in 
Sydney.  ADI  was  awarded  the  prime  contract  for  the  Huon  Class  minehunters,  based  on  an 
Italian  design  but  with  ADI  as  the  designated  design  authority  to  modify  and  Australianise 
the  design.  The  Aus$  917  million  (1994  prices)  project  was  the  first  Australian-sourced  naval 
project  in  which  the  local  prime  contractor  was  given  design  authority  (p.  67).  The  six  ships 
were  built  on  schedule  at  a  greenfield  site  facility  employing  new,  ‘greenfield’  labour  force 
(Parliament  of  Australia,  2006).  The  first  composite  hull  was  made  in  Italy  and  the  remaining 
five  in  Australia.  The  key  to  tight  schedule  success  was  an  onshore  facility  that  integrated 
and  tested  the  combat  system  prior  to  installation  (Tasman  Economics,  2002,  p.  9).  As  with 
ANZAC  frigates,  the  Huon  Class  also  complied  with  a  high  local  content  target  of  nearly 
70%. 


In  1999,  the  French  Thales  and  Australian  company  Transfield  bought  ADI  from  the 
Federal  Government  as  a  50-50  venture  (Parliament  of  Australia,  2006.  p.  71).  In  2006, 
Thales  Australia  was  granted  government  permission  to  acquire  the  Transfield’s  share  and 
consolidate  it  with  its  other  Australian  assets.  This  acquisition  has  turned  Thales  into  one  of 


59  During  one  of  the  authors’  visit  to  the  Williamstown  shipyard  in  the  mid-1990s. 
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Australia’s  largest  defence  contractors  and  a  key  naval  repair,  maintenance  and  upgrade 
contractor. 

FFG  upgrade  project 

In  contrast  with  the  very  successful  minehunter  project,  ADI’s  Aus$1  billion  upgrade 
of  four  FFGs  has  been  plagued  with  problems.  This  project,  commissioned  in  1999,  involves 
the  upgrade  of  ships’  combat  systems.  Initially,  it  was  to  cover  six  ships  but  as  the  first  ship 
was  delivered  three  years  late  (in  2006)  and  over  budget,  the  project  scope  was  reduced  to 
four  vessels.  The  upgrade  is  very  extensive  and  has  required  advanced  design  and 
engineering  work,  including  the  ADI-designed  and  developed  Australian  Distributed 
Architecture  Combat  System  (Parliament  of  Australia,  2006,  p.  69).  However,  “the 
Department  of  Defence  noted  that  while  ADI  is  viable  in  the  ship  repair  and  upgrade  activity, 
it  is  having  problems  in  meeting  schedule  and  performance  specifications”  (p.  69). 
Comments  such  as  this  cast  doubt  on  Thales’  chances  of  success  in  the  next  shipbuilding 
cycle,  even  though  in  2007,  it  was  Australia’s  largest  defence  contractor  (Hinz,  2007-2008). 

Armidale  patrol  boats  and  multihulls 

In  Australia,  there  are  two  relatively  small  but  internationally  competitive  commercial 
builders  of  aluminium  multi-hulls:  Austal  Ships  Ltd  (Austal)  and  Incat.  While  they  have  no 
experience  building  large  steel  vessels,  both  companies  have  established  market  niches  in 
wave  piercing  multi-hulls,  fast  multi-hull  ferries  and  luxury  motor  yachts.  Both  companies 
have  also  been  successful  exporters  and  are  well  regarded  internationally  for  their 
innovative  designs. 

In  2003,  Austal  won  an  Aus$553  million  project  to  build  14  Armidale  Class  patrol 
boats — the  last  major  contract  of  the  second  shipbuilding  cycle.  This  contract  was  innovative 
in  that  Defence’s  requirements  were  framed  in  terms  of  operational  performance 
specifications  (e.g.,  operational  availability)  rather  than  set  as  detailed  technical  guidelines 
for  ship  designers. 

In  2001,  Austal  also  opened  a  US  shipbuilding  facility  in  Mobile,  Alabama.  From  this 
foothold  in  the  US  shipbuilding  market  Austal  operates  as  part  of  the  General  Dynamics 
team  building  prototype  littoral  combat  ships  (LCS)  for  the  US  Navy.  Austal’s  role  is  to 
design  and  build  the  LCS  platform  for  USN.60  Austal  is  the  only  Australian  naval  shipbuilder 
to  be  involved  in  foreign  direct  investment  in  offshore  construction  facility  while  retaining  its 
core  design  team  in  Australia. 

In  the  early  2000s,  Incat  sold  and  leased  out  high-speed  catamarans  to  naval  users, 
including  the  Australian  Defence  and  the  US  Department  of  Defense.  However,  while  the 
adaptability  of  these  civil  ship  designs  to  military  uses  provides  an  example  of  dual¬ 
technology  opportunities  inherent  in  civilian  designs,  the  company  has  no  intention  to 
expand  its  operations  into  naval  shipbuilding  (Parliament  of  Australia,  2006,  p.  74).  Other 
small  shipbuilders  and  repairers  include  Forgacs  with  its  facilities  in  Newcastle  and 
Brisbane,  and  NQEA  based  in  Cairns. 


60  If  the  LCS  program  proceeds,  the  US  LCS  trimaran  project  may  involve  the  building  of  60  vessels 
at  a  cost  of  US$15  billion  (Parliament  of  Australia,  2006,  p.  72). 
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Third  post-war  cycle 

Based  on  the  2006  Defence  Capability  Plan  (DCP)  and  anticipated  upgrades  and 
maintenance,  Defence  intends  to  spend  about  Aus$30.5  billion  (2006  prices)  on  naval 
construction  and  sustainment  programs  between  2006  and  2025  (ADO,  2006b,  para.  3.4). 
While  the  proportion  of  local  content  differs  from  project  to  project,  about  Aus$19  billion 
(63%)  could  be  spent  in  Australia. 

At  the  start  of  the  third  construction  cycle  in  2007,  much  Defence  demand  for  naval 
construction  and  through-life  support  work  over  the  subsequent  period  was  committed  under 
supply  arrangements  already  in  place  or  soon  to  be  finalised.  These  included: 

■  the  sustainment  contracts  for  ANZAC  frigates  (Tenix  Marine  with  SAAB  as  the 
combat  systems  integrator),  Collins  Class  submarines  (ASC  with  Raytheon  as  the 
combat  systems  integrator),  and  Armidale  Class  patrol  boats  (Defence  Maritime 
Services); 

■  a  construction  contract  for  three  air  warfare  destroyers  (AWD)  awarded  to  ASC, 
based  on  the  Spanish  Navantia  design  (see  below),  and  a  contract  with  Raytheon  for 
the  AWD  combat  system,  which  is  likely  to  be  followed  by  a  future  contract  for 
through-life  support  with  the  two  companies;  and 

■  a  contract  for  two  landing  helicopter  dock  (LHD)  ships  awarded  to  Tenix  Marine, 
again  based  on  Navantia  design,  which  is  also  well  positioned  to  win  a  future 
contract  for  the  LHD  sustainment  support. 

The  early  commitment  of  such  a  large  proportion  of  the  2006-2018  spend  limits  the 
scope  available  to  Defence  to  attract  new  competition  into  the  domestically  located  market 
before  the  onset  of  the  fourth  shipbuilding  cycle  around  2018.  Although  the  support 
arrangements  for  the  AWDs  and  LHDs  are  yet  to  be  decided  when  their  construction  phase 
draws  to  a  close  (the  first  ships  are  expected  to  be  delivered  in  2012-2013),  the  logic  of 
Defence  sustainment  requirements  favours  the  existing  supplier  consortia. 

Also,  with  the  resource  export  boom  in  the  late  2000s  and,  thus,  tight  labour  markets, 
Defence  has  an  incentive  to  build  non-combatant  vessels  overseas.  At  an  international  level, 
competition  is  already  strong  and  the  competitiveness  of  the  market  could  be  reinforced  by 
the  availability  of  second-hand  civil  ships  that  could  be  adapted  locally  or  overseas  for 
Australian  use. 

Defence  continues  to  source  overseas  designs  for  its  major  platforms  (e.g.,  AWDs 
and  LHDs).  However,  past  schedule  slippages  and  cost  overruns  have  reduced  its  appetite 
for  extensive  Australianisation.  As  the  success  of  the  Spanish  Navantia  in  winning  the  AWD 
and  LHD  contracts  has  demonstrated,  overseas  shipbuilders  with  successful  designs 
adopted  by  a  foreign  parent  navy  will  be  able  to  compete  for  work  in  Australia  by  teaming 
with  Australian  prime  contractors.  Over  the  past  20  years,  this  preference  for  imported 
designs  has  produced  competition  between  design-based  consortia  of  shipbuilders, 
integrators  and  OEMs,  fronted  by  domestic  prime  contractors  but  also  including  overseas 
designers  and  suppliers.  This  form  of  competition,  and  the  increased  market  contestability 
resulting  from  the  threat  of  foreign  entry,  has  benefited  Defence  in  that  it  has  produced 
greater  market  rivalry  and  increased  scope  for  benchmarking  alternative  delivery 
arrangements. 
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Naval  Maritime  Industry 

Shipbuilders  and  ship  repairers 

The  traditional  concept  of  “naval  maritime  industry”  focuses  essentially  on  shipyard- 
based  shipbuilding  and  ship  repair/maintenance  activities.  In  this  narrowly  focused  approach 
to  defining  the  industry,  ship  assembly  and  module  manufacturing  are  included  as  long  as 
module  building  and  component  manufacture  are  undertaken  by  specialised  shipbuilders. 
Second-tier  suppliers  of  major  maritime  equipment  such  as  power  plants  or  navionics, 
normally  OEMs,  and  maritime  service  providers  such  as  naval  architects  and  surveyors  are 
also  included.  However,  jobbing  firms  supplying  components  made-to-order  are  likely  to  be 
excluded  as  are  most  third-tier  subcontractors. 

Another  distinction  has  traditionally  been  drawn  between  shipbuilding,  including 
capability  upgrades  and  ship  sustainment  (maintenance  and  repair,  including  battle  damage 
rectification).  These  two  sub-sectors  are  essentially  shipyard-based,  using  specialised 
infrastructure  such  as  dry  docks  and  sea  lifts.  In  Australia,  these  two  sectors  have  tended  to 
operate  in  parallel,  with  the  yards  involved  in  ship  repair  and  maintenance  separated  from 
those  used  in  shipbuilding  (e.g.,  the  Garden  Island  dockyard  specialising  in  ship  repair  while 
the  Williamstown  dockyard  is  used  to  integrate  new  vessels).  This  division  of  labour  has 
evolved  to  allow  platforms,  once  constructed  by  specialised  and  often  overseas-based 
shipbuilders,  to  be  maintained  and  repaired  by  “jobbing”  repair  yards  with  on-board 
equipment  supported  by  OEMs  and  jobbing  contractors.  This  division  of  labour  often 
required  long  supply  chains  linking  OEMs  to  maintenance  shipyards  and  led  to  delays  in  the 
availability  of  parts  and  long  repair  turnaround  times. 

Changes  to  the  traditional  division  of  labour  between  shipbuilding  and  ship 
repair/maintenance  were  driven  by  the  growing  complexity  of  platforms:  ships  were 
becoming  increasingly  automated,  requiring  the  integration  of  on-board  equipment  into 
larger,  network-based  and  knowledge-intensive  systems.  Sophisticated  ships  such  as 
modern  submarines  and  AWDs  are  increasingly  maintained  by  their  builders,  companies 
that  retain  the  IP  they  have  created  in  platform  design  and/or  work  closely  with  the  design 
authority  to  protect  and  support  the  integrity  of  ship  design.  The  retention  of  or  access  to 
design  IP,  the  use  of  dedicated  facilities  and  the  tacitness  of  ship-specific  knowledge  gained 
during  the  construction  phase  underpin  the  shipbuilders’  competitive  advantage  in  through- 
life  upgrade  and  maintenance  work.  Thus,  strong  synergies  (economies  of  scope)  have 
come  to  exist  between  the  construction  and  sustainment  phases  of  naval  capability.  Also, 
when  ships  are  built  in  small  batches  with  long  gaps  between  shipbuilding  cycles,  resources 
used  in  construction  (e.g.,  specialised  labour,  docking  facilities)  may  subsequently  be 
redeployed  in  fleet  sustainment. 

In  Australia,  this  synergistic  relationship  between  ship  construction  and  sustainment 
phases  was  first  exploited  in  the  Collins  Class  submarine  project:  the  Osborne  construction 
facility  in  South  Australia  is  dedicated  to  the  production  and  deep  maintenance  (full  docking) 
cycles  of  the  class.  However,  routine  maintenance  work  is  undertaken  in  Western  Australia, 
where  the  ships  are  home-ported.  This  model  of  “construction-enabled”  ship  maintenance 
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has  now  been  adopted  in  the  sustainment  of  other  vessels  (e.g.,  the  ANZACs)  and  is  also 
likely  to  be  used  in  support  of  future  additions  to  the  fleet  (e.g.,  AWDs  and  LHDs).61 

Australia’s  naval  shipbuilding  activity  is  largely  confined  to  four  main  shipbuilders: 
ASC,  Tenix  Marine,  Thales  (ADI)  and  Austal.  Of  these,  ASC  and  Austal  are  currently 
Australian-owned  while  Thales  is  a  fully  owned  subsidiary  of  the  French  parent  company 
and,  in  2008,  BAE  Systems  was  finalising  the  acquisition  of  Tenix  Marine.62  As  the  third 
post-war  building  cycle  began  to  unfold,  three  of  these  companies  were  involved  in  the 
construction  of  the  AWDs,  LHDs,  and  afloat  support  ships;  the  progressive  upgrades  of 
ANZAC  and  FFG  frigates,  the  Collins  Class  submarines,  minehunters  and  other  minor  war 
vessels;  and  maintenance  of  the  fleet-in-being.  Defence  Maritime  Services  (DMS)  are 
responsible  for  the  maintenance  of  Armidale  Class  patrol  boats  built  by  Austal.  (Some 
module  building  and  consolidation  work  and  maintenance  activity  has  been  undertaken  by 
smaller  maritime  suppliers  such  as  Forgacs,  with  facilities  in  Newcastle  and  Brisbane,  and 
NQEA  in  Cairns.)  The  three  shipbuilders  and  DMS  are  also  the  main  providers  of  naval 
sustainment  support  the  submarine  deep  and  intermediate  maintenance  cycles,  ANZAC  and 
FFG  frigate  sustainment,  support  for  minehunters,  patrol  boats,  and  other  minor  war 
vessels. 

Facility  disposition  and  ownership 

In  the  2000s,  Defence’s  preferred  industry  disposition  reflects  the  RAN’s  fleet  basing 
strategy,  which  envisages  the  maintenance  and  home-porting  of  major  surface  ships  on  the 
east  coast  of  Australia  (Sydney)  at  Fleet  Base  East  (FBE)  and  on  the  west  coast  (near 
Perth)  at  Fleet  Base  West  (FBW).  The  submarines  are  home  ported  and  maintained  at  FBW 
but  all  full-cycle  dockings  (deep  maintenance)  are  carried  out  in  South  Australia.  Minor  war 
vessels  are  mostly  home-ported  and  supported  in  Darwin  and  Cairns. 

The  home-porting  of  naval  vessels  at  FBW  has  spawned  the  development  of  navy- 
preparedness-related  industries  in  close  proximity  to  the  ships  they  support.  Thus,  in 
addition  to  major  shipbuilders  and  repairers  (e.g.,  Tenix  Marine,  ASC,  Austal),  other 
designers  and  builders  of  aluminium  boats  and  ships  and  engineering  firms  supporting 
resource  projects  have  clustered  in  Western  Australia,  in  particular  at  the  Australian  Marine 
Complex  (AMC)  in  Henderson.  There  appear  to  be  strong  agglomeration  economies  that 
naval  firms  can  gain  by  locating  at  AMC.  There  is  also  more  scope  for  forging  direct 
business  links  between  firms  that  operate  in  close  proximity. 

In  the  previous  naval  shipbuilding  cycle,  ownership  of  capital-intensive  facilities  (e.g., 
shiplifts  and  dry  docks)  was  a  key  characteristic  of  naval  shipbuilders.  This  is  still  largely  the 
case;  however,  the  high  cost  of  establishing  and  maintaining  such  facilities  constitutes  a 
formidable  barrier  to  entry  into  the  Australian  market  for  naval  shipbuilding  and  repair.  The 


61  Under  this  model  of  construction-enabled  ship  maintenance,  two  major  contracts  were  let.  In  2001 , 
Defence  signed  a  long-term  alliance  agreement,  underpinned  by  a  through-life  support  contract,  with 
Tenix  Marine  (shipbuilder)  and  SAAB  (system  integrator)  covering  the  development  of  all  future 
capability  change  packages  for  the  ANZAC  ships.  In  2003,  it  signed  another  long-term  contract  with 
ASC  for  the  25-year,  through-life  support  for  the  Collins  Class  submarines. 

62  However,  in  2007,  Tenix  Defence,  including  its  Tenix  Marine  Division,  was  offered  for  sale  and  BAE 
Systems  Australia  was  rumored  to  be  the  most  probable  buyer.  Also,  as  ASC  is  likely  to  be  offered  for 
sale  in  the  late  2000s,  Australian  subsidiaries  of  major  foreign  companies  may  be  invited  to  bid  for  it. 
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provision  of  these  facilities  involves  high-fixed  costs,  which  can  only  be  recouped  over  the 
long  term  and  which  even  the  largest  marine  companies  have  difficulty  absorbing  in  the 
relatively  small  Australian  market.  An  example  of  such  a  facility  is  the  shiplift/transfer  system 
operated  by  Tenix  Marine’s  facility  also  located  at  Henderson,  WA.  This  facility  was  initially 
funded  by  Defence  and  the  WA  State  Government  but  subsequently  sold  to  Tenix  Marine 
(Tenix,  2001).  Apparently  dissatisfied  with  this  arrangement,  the  West  Australian  (state) 
government  developed,  adjacent  to  the  Tenix  facility,  a  protected  deepwater  harbour — a 
15,000  tonne  service  and  heavy  lift  wharf,  and  several  other  facilities,  including  offices, 
workshops  and  other  amenities.  This  investment,  completed  in  mid-2003,  is  owned  by  the 
State  Government  and  operated  by  AMC  Management  (WA)  Pty  Ltd  as  a  common  user 
facility  (CUF).  While  Tenix’  Henderson  facility  is  maintained  by  the  company  for  its  own  use, 
the  CUF  is  deliberately  designed  for  multiple  users,  including  the  oil  and  gas,  resources, 
marine  and  defence  industries  and  is  sufficiently  large  to  accommodate  several  projects 
simultaneously.  Parties  using  the  facility  provide  their  own  management  and  workforce  and 
accept  normal  project  accountabilities.  They  use  the  CUF  only  when  their  projects  require  it 
and  are  charged  only  for  the  specific  facilities  they  use  for  a  particular  period.  This 
arrangement  greatly  reduces  project  set-up  costs  and  company  overheads,  thereby 
enhancing  CUF-users’  potential  ability  to  win  contracts. 

Initial  infrastructural  investments  in  the  Henderson  CUF  attracted  complementary 
private  investment  on  land  adjacent  to  the  marine  complex  (e.g.,  ASC  is  establishing  its 
submarine  maintenance  facility  there).  In  response  to  these  developments,  the  West 
Australian  government  invested  an  additional  Aus$81.5  million  in  a  floating  dock  to  launch 
and  dock  large  ships  and  a  rail  transfer  system  to  allow  construction  and  repair  within  the 
CUF’s  undercover  facilities;  an  extension  and  upgrade  of  the  existing  wharves  to 
accommodate  all  types  of  naval  and  commercial  vessels;  and  the  installation  of  marine 
services  such  as  power,  seawater  fire  main,  wharf  communications  and  sewerage  off-take. 

The  South  Australian  government  followed  suit  with  plans  to  invest  Aus$300  million 
in  Techport  Australia,  including  a  CUF  adjacent  to  ASC  in  Osborne  (Kerr,  2008b,  p.  2).  The 
SA  CUF  is  scheduled  for  completion  in  2010  and,  like  its  WA  counterpart,  is  intended  to 
support  multiple  projects  concurrently.  The  nearest  equivalents  to  such  infrastructure  on  the 
east  coast  are  the  Captain  Cook  Dock  (leased  by  the  Commonwealth  to  Thales  at  Garden 
Island,  Sydney). 

The  introduction  of  CUFs  funded  by  state  governments  and,  subject  to  leasing 
arrangements,  on-going  Commonwealth  ownership  of  the  Captain  Cook  Dock  combine  to 
reduce  the  significance  of  facilities  ownership  as  a  barrier  to  entry,  particularly  in  the  market 
for  naval  ship  repair.  As  an  indicator  of  policy  trends,  they  also  suggest  a  reappraisal  of  the 
value  of  public  ownership  of  assets  which  governments  were  so  determined  to  privatise  in 
the  late  1980s  and  1990s. 

Changing  division  of  labour 

The  impact  of  defence  procurement  on  industry  was  traditionally  viewed  in  terms  of 
the  relationship  between  an  agency  responsible  for  defence  procurement  and  the  prime 
contractors  with  which  it  negotiated.  These  days,  however,  it  is  recognised  that  effective 
procurement  depends  on  the  activities  and  performance  of  a  much  wider  range  of  industry 
players,  domestically  and  overseas. 
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Defence  considers  the  naval  maritime  industry  in  broad  terms  that  embrace  not  only 
shipbuilders  and  maintainers  but  also  a  myriad  of  second-  and  third-tier  SME  suppliers 
(ADO,  2006a,  paras.  1.18-1 .20).  The  latter  reportedly  account  for  some  70%  of  the  total  cost 
of  a  shipbuilding  project.63  As  noted  in  the  Defence  submission  to  the  2006  Senate  Inquiry,  a 
typical  frigate  comprises  some  170,000  parts  and  components  provided  by  600  suppliers 
and  sub-contractors  and  takes  1 .2  million  person-hours,  spread  over  22  months,  to 
construct.  A  large  conventional  submarine  may  consist  of  some  500,000  parts  provided  by 
1,600  suppliers  and  takes  2.5  million  person-hours  and  60  months  to  construct  (ADO, 

2006a,  Figure  1). 

Table  1  shows  the  stylized  breakdown  of  typical  warship  production  costs  that 
includes  all  on-board  combat  systems  but  excludes  capability  elements  that  are  shore, 
rather  than  ship-based.  In  the  table,  the  platform  element  of  capability  accounts  for  33%  of 
the  total  production  cost  for  a  more  technologically  complex  naval  combatant:  a  3,500  tonne 
frigate  costs  about  Aus$600  million  to  build,  while  on-board  combat  systems  account  for 
42%  of  the  cost.  The  other  two  cost  items  are  largely  platform-related  and  represent  the  cost 
of  logistic  support  acquired  during  the  construction  phase  and  the  cost  of  project  (delivery 
process)  management.  By  way  of  comparison,  for  a  large  naval  support  ship  constructed 
closer  to  commercial  standards,  on-board  combat  systems  account  for  only  15%  of  all  costs 
and  the  platform  for  47%  of  the  total.  For  a  naval  combatant  capability,  therefore,  the 
combat  systems  component  of  the  overall  system  is  the  most  important  element,  both  in 
cost  and  functional  terms.  This  is  reversed  in  the  case  of  the  naval  support  capability.  For  a 
typical  combatant  ship,  imported  combat  systems  and  other  major  equipment  account  for 
50%  of  the  construction  cost  (para.  2.5).  For  technologically  complex  vessels,  such  as  the 
submarines  and  the  AWDs,  the  proportion  is  likely  to  be  much  higher. 

Table  1  Percentage  Cost  Breakdown  in  Warship  Production 

(Based  on  ADP,  2006a,  Table  1,  para.  2.2) 


Production  cost  element 

Surface 

combatant 

ship 

% 

Support  ship 

% 

Platform  design,  hull,  machinery  and  equipment 

3 

47 

Combat  systems 

41 

15 

Logistics  support  and  training  (mostly  platform-related) 

17 

25 

Project  management 

9 

13 

Total 

100 

100 

In  Australia,  combat  systems  integrators  (e.g.,  Raytheon,  Thales,  BAE  Systems)  and 
OEMs  (e.g.,  STN  Atlas)  are  either  subsidiaries  or  agents  of  major  overseas  companies  (with 


63  These  suppliers  are  said  to  contribute  “70%  by  value  of  a  project”  (ADO,  2006a,  para.  1 .18). 
However,  the  third-tier  contractors,  as  well  as  other  tiers  of  suppliers,  should  not  be  seen  as  a 
reflection  of  a  hierarchical  industry  structure.  These  relationships  are  project-specific.  Thus,  a  large 
firm  that  is  engaged  as  a  prime  contractor  in  one  project  may  be  a  third-tier,  sub-contractor  in 
another. 
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the  notable  exception  of  CEA  Technologies).  In  the  early  2000s,  the  Australian  industrial 
footprint  of  these  multinational  companies  varied  from  significant  (Raytheon,  Thales)  to 
small  (Lockheed  Martin).  The  footprint  could,  in  most  cases,  be  flexibly  expanded  or  shrunk, 
depending  on  the  quantity  of  in-country  work  in  hand.  Much  has  been  claimed  by  these 
subsidiaries  for  their  direct  access  to  the  parent  company’s  global  network  and  technology. 
However,  Defence  has  at  times  observed,  “experience  indicates  that  they  have  difficulty 
obtaining  suitable  licensing  and  intellectual  property  rights  which  in  turn  may  have  time  and 
cost  implications  particularly  in  providing  sustainment”  (ADO,  2006a,  para.  2.9).  For  this 
reason,  the  Commonwealth  sometimes  facilitates  technology  transfers  using  government-to- 
government  arrangements  (e.g.,  the  US  FMS  framework)  to  secure  access  to  sensitive 
foreign  equipment,  military  technologies  and  IP  (e.g.,  the  direct  purchase  of  the  US  Aegis 
combat  system  for  the  AWDs  by  Defence  from  the  US  Navy  under  the  FMS  arrangement). 
Such  Commonwealth  action  has  direct  implications  for  the  role  of  prime  contractors,  an 
issue  we  address  below. 

Critical  to  the  provision  of  through-life  support  is  access  to  the  IP  behind  the  ship 
design.  At  the  smaller-vessel  end  of  the  naval  market,  Austal  is,  arguably,  the  only 
Australian  shipbuilder  offering  world  competitive  naval  design  expertise  for  multi-hull 
aluminium  vessels.64  For  larger  and/or  more  complex  ships,  Australia  has  been  an  importer 
of  ship  design,  usually  from  parent  navy  ship  designers  such  as  the  German  Blohm+Voss 
for  Meko  200  Class  frigates  (ANZAC  ships)  and  the  Swedish  Kockums  for  the  Collins  Class 
submarines.  However,  design  adaptation  to  meet  the  Australian  Navy’s  unique  requirements 
and  political  pressures  to  increase  local  content  have  resulted  in  considerable 
Australianisation  of  original  designs.  In  the  Collins  Class  case,  this  was  further  complicated 
by  the  transfer  of  ASC  ownership  to  the  Commonwealth.  It  was  only  when  the 
Commonwealth  negotiated  full  access  to  the  Kockums-owned  IP  that  ASC  became  the  de 
facto  design  authority  for  the  class  of  which  the  RAN  is  the  parent  navy.  Similarly,  Tenix 
Defence  is  the  de  facto  design  authority  for  ANZAC  ships.  The  Huon  Class  minehunter  was 
“the  first  Australian-sourced  naval  defence  project  in  which  the  prime  contractor  (ADI  now 
Thales)  was  given  design  authority”  (para.  4.39,  our  italics).  This  is  in  marked  contrast  to  the 
ANZAC  ship  and  Collins  Class  projects,  in  which  Tenix  and  ASC  effectively  became  design 
authorities  by  default. 

Defence  appears  to  be  determined  to  avoid  excessive  Australianisation  in  ship 
design  in  the  next  generation  of  vessels  to  be  constructed  in  Australia:  the  AWDs,  LHDs  and 
afloat  support  ships.  For  example,  in  the  case  of  AWDs,  the  government  overruled  Navy’s 
reported  preference  for  the  unproven  Gibbs  &  Cox  adaptation  of  the  Arleigh  Burke  destroyer 
in  favour  of  the  already-operational  Spanish  design  based  on  the  Navantia-built  FI  00 
destroyer  (Walters,  2007,  1  March,  p.  8). 

Marine  Industry  Supply  Chains 

Defence’s  broader  approach  to  what  constitutes  the  naval  marine  industry  has  also 
shifted  the  emphasis  from  functionally  based  naval  industry  sectors,  such  as  shipbuilders, 
OEMs  and  ship  repairers,  to  capability-centered  supply  chains  that  include  combat  systems 
integrators  and  the  plethora  of  second-  and  third-tier  suppliers,  many  straddling  sectoral 


64  This  is  reflected  in  its  aforementioned  involvement  as  a  ship  designer  and  potential  builder  in  the 
General  Dynamics-led  bid  for  the  US  Navy  Littoral  Combat  Ship. 
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divisions  and  serving  different  customers  in  different  industries.  While  the  functional 
representation  is  helpful  in  identifying  firms  largely  dedicated  to  shipbuilding  and  fleet 
sustainment,  the  supply  chain  framework  sheds  more  light  on  the  competitive  dynamics  of 
defence  capability  supply. 

Two  types  of  prime  contractor  arrangements  and,  thus,  supply  chain  structures,  have 
dominated  the  interface  between  Defence  and  Australian  naval  shipbuilders: 

■  a  traditional  single  channel  model,  under  which  a  single  prime  contractor  is  engaged 
by  Defence  to  lead  and  manage  the  supply  chain  and  to  orchestrate  all  the  back-to- 
back  contracts  with  upstream  suppliers  of  systems,  equipment,  components  and 
services;  and 

■  a  complex  multi-channel  model,  in  which  two  or  more  prime  contractors  are  engaged 
by  Defence  to  lead  and  manage  parallel  supply  channels  that  jointly  produce  the 
required  capability  element. 

■  These  two  models  are  used  both  in  shipbuilding  and  through-life  fleet  sustainment. 

To  illustrate,  consider  Figure  1,  in  which  two  stylized  traditional  supply  chain 
management  (SCM)  models  are  shown:  one  for  the  construction  of  a  support  ship  and 
another  for  a  major  weapons  upgrade.65  In  the  shipbuilding  case,  the  shipbuilder  is  also  a 
prime  contractor  who  engages  a  system  integrator  and  OEMs  as  well  as  a  large  number  of 
small  second-  and  third-tier  subcontractors  to  produce  the  end  product:  a  platform  with  all 
systems  and  equipment  integrated  into  it  or  on  it.  Although  the  ship  design  is  likely  to  be 
imported  and  Australianised,  it  is  a  relatively  simple  design.  Given  its  role  in  the  process,  the 
prime  contractor,  as  the  project’s  manager,  accounts  for  about  13%  of  the  total  project  cost. 
In  the  weapons  upgrade  case,  much  greater  weight  (and  cost  share)  is  assigned  to  combat 
system  integration  but  project  management  by  the  prime  contractor  still  accounts  for  about 
12%  of  the  total  project  cost. 


65  The  figure  has  been  stylized  using  total  project  cost  breakdown  by  project  elements  shown  in  ADO 
(2006a,  Table  1,  p.  8).  Thus,  the  cost  of  “platform  design”  is  imputed  to  Platform  Designer;  the  cost  of 
“combat  systems”  is  imputed  to  Combat  System  Integrator;  and  the  cost  of  “project  management"  to 
Prime  Contractor.  The  cost  of  “hull,  machinery,  equipment”  and  “logistics  support,  including  training” 
is  attributed  to  Shipyard  (operator)  and  OEMs.  Other  second-  and  third-tier  suppliers  are  included  in 
OEM,  Shipyard  and  Combat  Integration  cost  elements. 
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Figure  1  Conventional  Value  Chains  for  Support  Ship  and  Weapons  Upgrade: 
Total  Project  Cost  Breakdown  by  Supplier  Category 

(Based  on  ADO,  2006a,  Table  1,  p.  8) 


Upstream  activities  Downstream  activities 

Second  and  OEMs  Shipyard  Combat  System  Platform 

third  tier  Integrator  Designer 

suppliers  ^ ^ - J 

65%  15%  7% 


End  Customer 
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Contractor 
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TTTT.j..|...|  Defence/Navy 


SUPPORT  SHIP  CONSTRUCTION 


35%  50%  3%  12% 


WEAPONS  UPGRADE 


Defence/Navy 


The  conventional  prime  contracting  model  has  traditionally  been  used  by  Defence  as 
a  risk  management  arrangement  under  which  the  prime  contractor  is  expected  to  manage 
and  mitigate  risks  associated  with  the  operation  of  the  supply  chain.  This  model  was  used 
by  Defence  during  the  second  building  cycle  in  all  major  shipbuilding  projects,  initially 
including  the  Collins  Class  submarines.  But  the  conventional  model  failed  the  test  when  the 
Collins  Class  project  ran  into  problems  with  combat  system  integration.  By  1993,  Rockwell, 
the  combat  systems  integrator,  was  not  able  to  comply  with  Navy’s  specifications  and  “ASC 
effectively  lost  control  of  the  Rockwell  sub-contract”  (Parliament  of  Australia,  2006,  para. 
4.18).  As  noted  earlier,  the  solution  involved  replacing  the  original  combat  system  integrator 
and  Defence  awarding  the  contract  to  Raytheon  in  2001  (Parliament  of  Australia,  2006). 
Under  this  arrangement,  Raytheon  became  a  parallel  prime  contractor  for  system 
integration.  To  complicate  the  model  further,  “Defence  itself  has  essentially  primed’  the 
subsequent  Aus$500  million  combat  data  system  replacement  program  by  purchasing  the 
FMS-mediated  software  and  working  with  ASC,  Raytheon,  Atlas  Electronics  and  Thales 
Underwater  Systems  to  integrate  all  combat  systems  (para.  4.20,  our  italics). 
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Figure  2  Complex  Supply  Chain  for  a  Naval  Combatant:  Total  Project  Cost 

Breakdown  by  Supplier  Category 

(Based  on  ADO,  2006a,  Table  1,  p.  8) 
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The  resulting  structure  is  represented  in  Figure  2,  which  shows  a  complex,  multi¬ 
channel  supply  chain  (say,  for  a  frigate-type  naval  combatant).  In  the  figure,  the  stylized 
supply  chain  involves  two  parallel  channels  of  progressive  value-adding  activity:  platform 
construction  and  systems  integration.  The  figure  highlights  downstream  activities  (close  to 
the  end  customer)  such  as  project  management,  design,  and  platform  integration  along  the 
platform  construction  supply  channel  and  combat  system  integration  along  the  systems 
integration  channel.  Further  upstream  are  OEMs  that  provide  equipment  and  subsystems  for 
downstream  platform  and  systems  integrators  and  other  second-  and  third-tier 
subcontractors  who  provide  inputs  for  OEMs  and  downstream  integration  activities.  Some  of 
these  smaller  second-  and  third-tier  contractors  are  specialised  naval  suppliers  but  most 
tend  to  be  broadly  based  manufacturers  and  service  providers.  Also,  some  apparently  small 
subcontractors  (in  terms  of  quantities  and  dollar  value  of  supplies)  are  subsidiaries  or  agents 
of  large  producers  of  generic  products.  As  we  move  from  right  to  left  along  each  supply 
channel,  from  downstream  to  upstream  activities,  suppliers  are  less  likely  to  be  dedicated  to 
the  production  of  naval  systems.  The  reduced  role  of  the  prime  contractor  for  the  platform  is 
indicated  by  the  smaller  proportion  of  the  total  project  cost  (9%). 

This  representation  of  the  naval  construction  supply  chain  for  complex  projects 
emphasises  the  changing  concept  of  the  prime  contractor.  In  this  case,  there  are  two  prime 
contractors  operating  in  parallel,  the  shipbuilder  (prime  contractor  for  the  platforms)  and  the 
systems  integrator  (prime  contractor  for  the  combat  system).  Shipbuilding  activity  accounts 
for  nearly  60%  of  the  total  project  cost  and  systems  acquisition  and  integration  for  over  40%. 
The  figure  highlights  an  important  aspect  of  complex  naval  ship  construction:  the 
management  of  the  multi-channel  supply  chain  is  distributed  between  two  or  more  prime 
contractors,  each  responsible  for  the  orchestration/management  of  construction/integration 
activities  along  its  particular  supply  channel.  This  at  once  raises  a  higher-level  coordination 
problem:  Defence,  through  its  procurement  agency  DMO  (shown  in  Figure  2  as  a  “capability 
prime”),  is  now  responsible  for  coordinating  the  activities  of  the  two  (channel-specific)  prime 
contractors.  This  necessarily  implies  that  Defence  cannot  (as  it  has  often  sought  to)  adopt 
and  maintain  an  arms-length  relationship  with  its  suppliers.  The  new  model  has  already 
been  applied  in  the  acquisition  of  the  AWDs  via  an  alliance-based  contracting  strategy 
(Australian  DoD,  2008).  This  strategy  is  given  practical  effect  through  an  Alliance-based 
Target  Incentive  Agreement  signed  in  October  2007  by  the  Defence  Materiel  Organisation, 
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ASC  as  the  designated  builder  and  prime  contractor  for  the  Navantia-designed  AWD 
platform  and  Raytheon  Australia  as  the  combat  system  integrator.66  Defence  is  also  directly 
involved  in  the  supply  chain  management  as  it  purchased  directly  from  the  US  Navy  the 
Aus$1  billion  US  Lockheed  Martin  Aegis  combat  system,  which  Raytheon  is  to  integrate  with 
the  platform  and  other  on-board  systems. 

Competition  for  Large,  Complex  Projects 

During  the  second  shipbuilding  cycle,  the  competitive  conduct  of  defence  naval 
suppliers  was  assigned  a  pivotal  role  in  achieving  “value  for  money”  for  the  Commonwealth 
and  became  a  mantra  of  Defence  procurement.  Competition  to  take  on  the  role  of  prime 
contractor  for  larger,  complex  naval  projects  took  the  form  of  rivalry  among  consortia  formed 
between  Australian  shipbuilders,  overseas  designers,  and  Australian  subsidiaries  of 
overseas  systems  houses.  The  competitive  process  led  to  the  award  of  contracts  to 
successful  consortia  (e.g.,  AMEC-Blohm+Voss  for  the  ANZAC  Ships)  using  the  conventional 
model  of  engagement  between  the  prime  contractor  and  Defence.  This  mode  of 
engagement  had  worked  reasonably  well  for  projects  involving  less  complex  deliverables 
(e.g.,  the  ANZAC  ships).  However,  as  the  experience  of  the  Collins  project  demonstrated, 
the  conventional  model  based  on  the  arm’s-length  relationship  between  Defence  and  prime 
was  not  suitable  for  procuring  complex  capabilities  such  as  submarines  or  technologically 
challenging  systems  upgrades  (re:  the  troubled  FFG  upgrade).  A  key  reason  for  the  difficulty 
lies  in  elements  of  hold-up  present  in  the  relationship  between  the  incumbent  prime 
contractor  and  Defence. 

While  competition  is  normally  used  to  engage  prime  contractors  fronting  competing 
overseas  designs,  competitive  pressure  on  prime  contractors,  combat  systems  integrators 
and  often  key  OEMs  tends  to  fall  away  once  the  prime  contract  is  signed.  If  the  prime 
contractor  fails  to  deliver  contracted  performance,  slips  behind  schedule,  or  runs  over  the 
budget,  Defence  is  heavily  constrained  in  its  option  for  remedial  action.  Switching  prime 
contractors  and/or  main  sub-contractors  is  often  technologically  infeasible,  financially 
prohibitive  or  politically  embarrassing.  Even  for  a  medium  size  naval  project,  such  as  the 
FFG  upgrade,  the  prime  contractor  was  allowed  to  continue  with  the  project,  despite  public 
expressions  of  dissatisfaction  from  the  client  and  an  adverse  national  audit  report.  Despite 
tough  rhetoric  in  public,  Defence  has  only  limited  scope  to  bring  a  contractor  into  line.  A 
financial  penalty  for  contractual  default,  for  example,  may  be  no  more  than  a  slap  with  the 
business  equivalent  of  a  feather.  And,  when  the  worst  comes  to  the  worst — as  in  the  case  of 
Rockwell’s  failure  to  deliver  the  CDS  system  for  the  Collins  Class  submarines — Defence 
decided  against  re-competing  the  requirement  and,  instead,  appointed  a  substitute, 
Raytheon,  to  take  over  as  system  integrator. 


66  The  arrangement  takes  the  form  of  Alliance-based  Target  Incentive  Agreement  (ABTIA)  between 
the  Commonwealth,  represented  by  the  Defence  Materiel  Organisation,  ASC,  as  the  shipbuilder  and 
Raytheon  Australia  as  the  mission  systems  integrator.  “The  broad  AWD  procurement  principles 
articulated  by  the  Alliance  comprise  value  for  money,  efficient  and  effective  process,  ethics  and 
probity,  accountability  and  transparency,  good  faith  and  fair  dealing  and  competition ”  (Kerr,  2008b, 
pp.  2-3;  our  italics).  Under  this  arrangement,  major  equipment  is  already  specified  by  Navantia  SA — 
the  Spanish  designer — and  the  Alliance  will  utilise  Navantia’s  established  supply  chain.  Navantia  will 
perform  all  the  required  design  modifications  and  will  maintain  design  configuration  control.  Raytheon 
will  undertake  the  Australianisation  of  the  combat  system  around  the  fully  imported  Aegis  core 
sourced  by  the  Commonwealth  via  the  US  FMS  (2008b,  pp.  2-3). 
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In  the  third  naval  procurement  cycle,  much  of  the  competitive  process  for  major  naval 
projects  was  completed  early  on,  with  winning  consortia  announced  for  AWDs  and  LHDs 
and  large,  long-term  support  contracts  for  ANZACs  and  Collins  submarines  given  to  Tenix 
and  ASC.  It  seemed  unlikely  that  any  prime  contractor  would  subsequently  be  dumped  and 
replaced  by  another  contractor.  However,  there  was  a  key  difference  between  the  second 
and  the  third  shipbuilding  cycles.  Defence  had  become  aware  that  lack  of  effective 
competitive  pressure,  following  contract  award  had  deprived  it  of  effective  market  power  vis- 
a-vis  its  larger  prime  contractors.  The  complex,  multi-channel  procurement  model  described 
above  can  be  viewed  as  an  evolutionary  adaptation  responding  to  Defence’s  recognition 
that,  to  maximise  the  likelihood  of  success  for  its  projects,  it  would  have  to  embrace  fully  its 
ultimate  responsibility  as  “prime  contractor  of  the  last  resort.”  It  had  been  obliged  to  accept 
that  the  prospects  for  shifting  project  risk  to  primes  in  large,  strategically  important  projects 
were  at  best  limited  and,  realistically,  often  unachievable.  For  successful  outcomes,  Defence 
would  have  to  manage  projects  more  proactively  and  build  close,  synergistic  relationships 
with  primes  rather  than  relying  on  contract  specifications,  impracticable  penalties  for  non¬ 
performance  and  arm’s-length  dealings  with  contractors. 

Lessons  Learnt 

In  a  microcosm  of  naval  shipbuilding  activity,  this  case  study  shows  in  detail  how 
various  defence  industry  policies  and  procurement  initiatives  have  worked  in  Australia  over 
the  past  30  years.  By  focusing  on  naval  shipbuilding  and  repair,  we  have  not  only  selected  a 
sector  that  is  seen  in  Australia  as  the  jewel  in  the  defence  industry  crown  but  also  one  that 
comprises  a  wide  variety  of  business  entities — from  diversified  large  contractors  to  highly 
specialised  small  firms,  including  new  forms  of  government  shipyards  such  as  CUFs  and 
technologies  such  as  mechanical  (platforms),  IT  (combat  systems,  specialised  equipment). 
We  conclude  this  paper  by  highlighting  what  are,  in  our  view,  key  lessons  to  be  drawn  from 
Australian  naval  shipbuilding  experience.  These  concluding  comments  focus,  respectively, 
on  the  demand  and  supply  sides  of  the  market  and  the  demand-supply  interface. 

Demand 

Lumpiness  of  demand 

For  reasons  associated  with  durability,  cost  and  changes  in  military  technology,  most 
defence  systems,  including  those  embedded  in  naval  capabilities,  tend  to  be  replaced  at 
relatively  widely  spaced  intervals  rather  than  continuously.  This  applies  to  simple  weapons 
systems,  such  as  small  arms,  but  particularly  to  large  and  chunky  elements  of  capability 
such  as  naval  ships  that  tend  to  be  replaced  as  fleets.  This  batching  of  demands  can  be 
smoothed  by  Defence  to  the  extent  that  fleet  replacements  can  be  staggered,  but  some 
lumpiness  of  demand  seems  unavoidable.  Long-term  forward  plans,  such  as  the  Australian 
DCP,  make  it  easier  for  industry  to  anticipate  forthcoming  demand  and  ramp  up  for  future 
tenders.  But  the  scope  for  demand  smoothing  is  limited  as  Defence  has  to  be  flexible 
enough  in  its  forward  commitments  to  respond  to  changing  strategic  and  economic 
circumstances,  sometimes  at  very  short  notice. 

Asset  ownership 

The  complete  control  of  “use  rights”  is  necessary  for  key  combat  assets  such  as 
naval  combatants.  This  can  be  achieved  through  the  conventional  full  ownership  of  ships,  or 
through  leasing  arrangements,  particularly  the  leasing  of  vessels  from  foreign  governments. 
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However,  more  flexible  arrangements  can  be  used  to  procure  the  services  of  secondary 
assets  such  as  patrol  boats,  which  tend  to  operate  in  peacetime  in  more  stable  roles  and 
predictable  circumstances.  As  demonstrated  by  the  Armidale  patrol  boat  arrangement,  the 
procurement  of  ship  services  from  a  private  consortium  of  maritime  service  providers  rather 
than  the  full  ownership  of  vessels  is  feasible  and  attractive.  And,  in  the  event  of  war,  the 
nature  of  the  relationship  can  be  changed  by  placing  the  vessels  under  complete  naval 
control. 

Local  content  requirements 

The  history  of  Australian  naval  shipbuilding  and  repair  highlights  often-encountered 
trade-offs  between  local  and  overseas  sources  of  supply  and  naval  preparedness.  In 
Australia,  as  in  other  small  countries,  it  is  increasingly  accepted  that  building  ships  in 
country  is  politically  as  much  as  strategically  driven.  In  modern  warfare,  there  is  no  time  to 
replace  combat  assets,  such  as  ships,  and  nations  are  unlikely  to  engage  in  wars  of 
equipment  attrition.  It  is  thus  perceived  as  more  important  to  have  domestic  industry 
capability  on  hand  to  undertake  ship  repair  and  modification,  including  battle  damage 
rectification.  The  LHDs  project,  incorporating  hull  construction  at  Navantia’s  Ferrol  shipyard 
in  Spain  and  superstructure  by  Tenix  in  Australia,  departs  from  the  recent  tradition  of 
building  ships  in-country  to  an  imported  design.  However,  the  procurement  of  the  AWDs 
follows  the  conventional  path,  with  expectations  that  substantial  premia  will  be  paid  for  the 
political  decision  to  construct  them  in  South  Australia  (Dodd,  2008). 

Australia  continues  to  import  naval  ship  designs  and  the  recent  tendency  is  to 
minimise  design  Australianisation  (Kerr,  2008a;  2008b).  To  reduce  risks  of  “design 
parentage,”  the  approach  is  to  incorporate  MOTS  components  in  imported  systems  and 
make  considerable  use  of  the  design  authority’s  established  supply  chain  (e.g.,  the  AWD 
and  LHD  arrangements  with  Navantia).  Political  pressures  are  likely  to  support  ongoing  high 
levels  of  local  content  in  platform  construction,  but  strategic  issues  may  be  more  important  in 
influencing  levels  of  local  content  in  combat  systems  maintenance  and  modification. 

Defence  may  be  worried  about  the  risk  of  relying  on  local  supply  for  developmental 
components.  For  example,  the  locally  developed  CEAFAR  active  phased  array  radar  for  the 
AWDs  has  not  been  included  in  the  baseline  specification  as  it  is  still  under  development  by 
its  maker — a  small  but  high  profile  Australian  firm  CEA.  But,  the  new  technology  is  likely  to 
be  incorporated  as  it  matures  (Kerr,  2008b). 

Business  models 

A  range  of  new  business  models  has  evolved  in  Defence  to  engage  suppliers  in  the 
most  effective  way.  These  models  tend  to  be  tailored  to  the  nature  of  the  product  and  the 
characteristics  of  the  supplier.  When  mature  products  are  supplied  by  established 
contractors,  the  inherent  risks  of  performance  degradation  and  schedule  slippages  are  low 
and  traditional  fixed/firm  price  models  can  be  used.  An  evolved  model  of  this  kind  has  been 
used  to  acquire  the  services  of  the  Armidale  Class  patrol  boats.  When  the  developmental 
content  of  the  product  increases  and  if  the  supplier’s  track  record  also  inspires  less 
confidence,  various  forms  of  incentive  and  incremental  contracting  are  more  likely  to  be 
used  (e.g.,  the  acquisition  of  the  electronic  warfare  system  for  the  AWDs — Kerr,  2008b,  p. 

6).  And,  for  technologically  sophisticated,  complex  and  politically  high-profile  acquisitions, 
such  as  the  AWDs,  it  is  now  accepted  that  the  Commonwealth  cannot  divest  itself  of  its 
ultimate  responsibility  for  strategic  capability  formation.  In  the  emergence  of  the  multi-prime 
contractor  model  (in  which  DMO  has  entered  a  “prime  alliance”  with  shipbuilder  and  systems 
supplier),  it  has  been  recognised  that  the  buyer’s  procurement  agency  must  engage  in 
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relationship  management  and  that  even  a  very  detailed  contract  cannot  shift  all  risks  away 
from  the  Commonwealth  to  commercial  prime  contractors. 

Supply 

Facility  ownership 

It  was  widely  claimed  in  the  1980s  and  1990s  that  privatising  government  shipyards 
and  factories,  and  related  ‘private  finance  initiatives’,  was  a  necessary  precondition  for  their 
improved  productivity  and  dependability.  In  2000s,  however,  it  became  increasingly 
apparent  that  the  private  sector  would  not  invest  in  capital  intensive  assets  such  as 
shipyards  unless  it  could  reasonably  expect  an  adequate  return  on  its  investment. 
Commercial  owners  would  only  invest  in  new  shipbuilding  facilities  if  their  order  books 
justified  the  heavy  capital  commitment.  This  in  turn  depended  on  owners’  confidence  in  a 
continuing  flow  of  potentially  profitable  orders  -  hard  to  create  in  the  face  of  a  history  of  long 
intermissions  in  demand  and  the  competitive  processes  for  allocating  work.  It  has,  thus, 
been  recognised  that  competitive  sourcing  might  have  to  be  abandoned  in  favour  of  sole  or 
dual  sourcing  if  local  platform  builders  and  maintainers  are  to  be  encouraged  to  invest  in 
capital-intensive  facilities.  But,  it  also  seems  increasingly  accepted  that  lack  of  competition 
and  not  the  public  ownership  per  se  was  the  main  cause  of  poor  performance  of  government 
shipyards  and  factories.  The  designation  of  a  private  contractor  as  sole  source  provider  to 
Defence  is  likely  to  lead  to  many  problems  previously  experienced  with  government-owned 
enterprise. 

As  the  third  post-war  building  cycle  unfolded,  the  competition  between  the  States  for 
defence  orders  has  resulted  in  renewed  public  investment  in  capital  infrastructure  in 
shipbuilding  and  repair  (re:  the  CUF  model  was  pioneered  by  Western  Australia  and 
adopted  by  South  Australia).  And,  the  Commonwealth  of  Australia  (federal  government)  has 
retained  its  ownership  of  the  Garden  Island  dock  leased  to  Thales.  Under  CUF 
arrangements,  governments  attract  and  sustain  private  naval  investment  by  investing  in 
complementary  infrastructure  and  engaging  in  a  form  of  quasi-vertical  integration  under 
which  the  publicly  owned  asset  is  then  leased  to  a  private  contractor  for  the  period  it 
requires  to  supply  goods  and  services  to  Defence. 

The  1990s  and  2000s  have  also  witnessed  increased  penetration  of  the  Australian 
shipbuilding  sector  by  overseas  capital.  Of  the  three  largest  naval  shipbuilders,  two  (ADI  and 
Tenix)  have  become  subsidiaries  of  foreign  companies  (Thales  and,  subject  to  satisfactory 
negotiations,  BAE  Systems,  respectively).  The  third,  ASC,  is  to  be  sold  in  the  late  2000s, 
and  may  yet  end  up  in  overseas  ownership.  All  systems  integrators  and  nearly  all  major 
OEMs  (except  CEA)  are  subsidiaries  of  overseas  companies.  And  P&O  Maritime  Services 
and  Serco  Australia  have  pioneered  the  provision  of  fully  supported  services  for  minor  naval 
vessels  (the  Armidales).  This  trend  is  very  much  in  keeping  with  global  developments  in 
defence  industry.  Few  small  countries  can  support  indigenous  systems  integrators  and 
OEMs  while  exporting  of  defence-related  products  from  small  countries  poses  well-known 
difficulties. 


Structure 


The  1990s  and  2000s  have  also  seen  the  increased  consolidation  of  ship  assembly 
in  fewer  hands  and,  in  a  clear  break  with  the  past,  a  growing  integration  between 
shipbuilding  and  repair.  In  part,  the  latter  trend  reflects  the  changing  global  division  of  labour 
as  systems  houses  and  OEMs  become  increasingly  involved  in  the  provision  of  through-life 
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support  for  their  products.  In  part,  it  also  reflects  the  shift  of  emphasis  in  Australian  industry 
policy  from  a  focus  on  platform  construction  to  through-life  capability  support.  The  three 
large  projects  of  the  second  shipbuilding  cycle  have  also  had  implications  for  the  size 
distribution  of  firms  in  the  sector.  The  building  of  the  ANZACs,  Collins  submarines  and  Huon 
minehunters  attracted  a  large  number  of  firms  to  third-tier  naval  subcontracting.  As  a  result, 
the  size  distribution  takes  a  Pareto  form  with  a  small  number  of  large  naval  firms  operating 
in  the  first  and  second  tiers,  downstream  in  the  supply  chain;  a  large  number  of  third-tier 
sub-contractors  are  engaged  in  the  upstream  segments  of  the  chain. 


Conduct 


In  the  1990s  and  2000s,  firms  have  increasingly  made  efforts  to  collaborate  along 
the  supply  chain  rather  than  to  do  business  with  each  other  at  arm’s  length.  On  the  other 
hand,  firms  also  appear  to  have  been  competing  with  increasing  frequency  and  intensity  for 
markets  opening  up  for  all  segments  of  the  naval  supply  chain.  Firms’  awareness  of  their 
mutual  dependence  in  the  network  of  supply  arrangements  appears  to  be  driving  a  tendency 
to  greater  collaboration  once  the  principal  contract  has  been  awarded. 

Performance 

The  second  shipbuilding  cycle  saw  a  marked  increase  in  shipyard  productivity  and 
less  severe  budget  overruns  and  schedule  slippages.  The  Collins  Class  project  was  the 
most  troublesome  acquisition  of  the  period  but  by  no  means  because  of  problems  restricted 
to  the  supply  side.  The  FFG  upgrade  project  appears  to  have  suffered  from  the  classic 
syndrome  of  supplier  overconfidence:  ADI  seems  to  have  lacked  awareness  of  its  capability 
limitations  and  underestimated  the  importance  of  technical  challenges  that  were  likely  to 
arise  in  a  project  of  such  complexity.  As  the  third  shipbuilding  cycle  unfolded,  Defence  (and 
indirectly  the  government)  appeared  reluctant  to  risk  quality-budget-schedule  outcomes  by 
trying  to  over-Australianise  designs  and  aiming  at  high,  local-content  targets.  “Buying 
MOTS”  and  minimising  the  local  developmental  content  characterised  its  strategy  to  head 
off  poor  performance. 

Demand-supply  interface:  Competition  for  and  in  the  market 

Arguably  the  most  striking  development  since  the  end  of  the  second  shipbuilding 
cycle,  particularly  in  the  aftermath  of  the  Collins  project,  has  been  much  better 
understanding  by  Defence  of  competitive  processes,  especially  the  difference  between  /br¬ 
and  in- the  market  competition.  This  reflects  the  growing  maturity  of  Defence  as  an  investor 
in  new  capability  elements  and  buyer  of  military  materiel.  It  is  increasingly  accepted  that 
different  competitive  processes  operate  for  different  segments  of  the  naval  supply  chain. 
Creating  a  competitive  environment  in  downstream  segments  of  the  chain  in  a  small  country 
calls  for  opening  the  market  to  overseas  participants.  When  this  is  done,  the  range  of 
competing  designs  and  combat  systems  is  broadened  as  overseas  consortia  of  platform 
builders,  system  integrators  and  OEMs  (sometimes  combining  with  local  firms)  come  to 
contest  the  market.  Once  a  preferred  package  has  been  selected,  competition  in  the  market 
follows,  and  sub-contractors  vying  for  various  elements  of  the  package  emerge.  Domestic 
subcontractors  can  be  assured  a  major  role  in  this  part  of  the  process  if  local  content 
requirements  are  in  force. 

Finally,  it  appears  that  Defence  has  become  more  aware  of  the  difference  between 
the  pre-  and  post-contract  opportunities  open  to  it  in  sourcing  supplies.  That  is,  it  is  better 
understood  that,  once  the  contract  is  signed,  switching  suppliers  and  supplies  may  be 


ACQUISITION  RESEARCH:  CREATING  SYNERGY  FOR  INFORMED  CHANGE  -  384  - 


impossible  for  technological,  budgetary  or  political  reasons.  As  a  project  progresses  through 
the  tendering  process,  the  scope  for  product  and  supplier  substitution  decreases  and,  for 
major  projects,  there  may  be  no  realistic  way  of  returning  to  status  quo  ante.  Post  contract, 
competition  in  the  market  is  largely  restricted  to  upstream  segments  of  the  supply  chain  in 
which  third-tier  suppliers  are  easier  to  replace  if  their  contractual  performance  is  inadequate. 

Also,  applying  a  one-size-fits-all  business  model  can  often  be  a  recipe  for  failure  in 
defence  procurement.  But,  to  tailor  different  models  to  different  acquisitions,  it  is  necessary 
to  acquire  good  understanding  of  supply  conditions  and  commercial  business  processes. 
Following  its  designation  as  a  prescribed  agency,  the  DMO  has  become  increasingly 
professionalised  as  a  procurement  agency  and  as  a  hands-on  equity  partner  in  major 
acquisition  projects. 

Conclusion 

Faced  with  the  challenge  of  efficiently  procuring  naval  vessels  of  increasing 
technological  sophistication,  the  Australian  government  has  learned  over  recent  decades 
that  contract  arrangements  alone  are  often  insufficient  to  allow  it  to  address  and  remedy 
problems,  especially  when  developmental  issues  are  at  stake.  While  fixed-cost  contracts,  for 
example,  apparently  offer  the  Defence  customer  the  prospect  of  shifting  all  risk  to  its 
industry  suppliers,  the  experience  of  the  Collins  Class  submarine  clearly  showed  that  when 
the  success  of  the  project  was  seriously  threatened,  the  government  felt  it  had  little  option 
but  to  intervene  directly  to  re-organise  supply-side  production  arrangements.  As  the  nature 
of  the  naval  warship  has  changed  with  technological  innovation,  it  has  also  become  clear 
that  government  must  take  on  an  overarching  prime  responsibility  if  the  production  tasks 
involved  are  to  be  effectively  coordinated.  A  warship  is  a  sea-borne  platform  carrying 
weapons.  But  the  business  of  designing  and  building  sea-worthy  and  battle-ready  vessels  is 
altogether  different  from  the  enterprise  of  designing  and  producing  the  highly  sophisticated, 
often  network-integrated  weapons  systems  that  the  warship  must  support.  We  have  shown 
that  the  Australian  government  has  recognised  the  force  of  this  reality  by  creating  different 
industry  primes  for  platform-building  and  weapons  production  and  adopting  the  coordinating 
role  for  itself.  Despite  past  rhetoric  to  the  contrary,  innovation  and  complexity  in  design  and 
production  appear  to  create  conditions  in  which  governments  find  themselves  obliged  to 
form  close  and  durable  relationships  with  suppliers  if  they  wish  to  maximise  the  likelihood  of 
project  success.  It  may  neither  be  realistic,  given  the  industry  structure,  nor  wise,  given  the 
alternatives  available  to  suppliers,  for  governments  to  threaten  competitive  recontracting  as 
their  sole,  or  even  principal,  means  of  discipline  and  performance  control. 

For  political  reasons  familiar  in  most  countries  (smaller  ones  being  no  exception), 
governments  routinely  find  themselves  under  pressure  to  favour  domestically  located 
supply.  If  the  depth  and  breadth  of  expertise  and  capabilities  in  local  defence  industry  is 
limited,  there  is  the  potential  of  conflict  with  the  goals  of  successfully  procuring  increasingly 
sophisticated  systems,  especially  if  tailored  idiosyncratically  to  national  requirements.  As  this 
paper  has  shown,  Australia  has  at  times  focused  heavily  on  local  content  requirements  in 
naval  shipbuilding  and,  whatever  the  benefits,  has  sometimes  paid  a  high  cost  premium  for 
doing  so.  The  issues  around  such  requirements  are  not  likely  to  disappear  in  Australia  or  in 
other  countries  in  the  foreseeable  future. 

In  relation  to  naval  ships,  it  may  appear  eminently  sensible  and  potentially  efficient  to 
provide  sustainment,  repair  and  maintenance  for  warships  domestically  but  more 
problematic  to  justify  actually  building  the  vessels  in-country.  On  the  other  hand,  it  can  be 
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argued  that  such  a  large  fraction  of  through-life  costs  relate  to  post-delivery  support  that  any 
cost  premium  on  domestic  construction  can  be  discounted  as  relatively  unimportant.  If  local 
through-life  support  is  more  efficient  when  ships  are  also  built  locally  in  the  first  place,  the 
argument  is  reinforced.  Analogous  arguments  may  also  be  applied  to  other  sorts  of  platform 
and  weapons  system.  No  simple  generic  solution  indicates  when  make  domestically  should 
be  preferred  to  buy  overseas  in  such  cases.  But  the  historic  experience  of  substantial  cost 
premia  on  local  content  in  small  country  environments  suggests  that  a  critical  eye  should 
always  be  applied  to  ex  ante  predictions  of  large  expected  net  benefits  from  locally 
producing  the  more  innovative  and  idiosyncratic  weapons  systems. 
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Abstract 

With  the  rapid  evolution  of  the  global  defense  management  environment,  an 
innovative  strategy  to  reinforce  the  international  competitiveness  of  the  National  Defense 
Industry  (NDI)  through  competitiveness  analysis  is  required.  Thus,  it  is  necessary  to  develop 
appropriate  models  for  analyzing  the  Korean  NDI  and  to  conduct  competitiveness  analysis 
by  using  the  developed  model. 

For  the  purpose  of  analyzing  the  international  competitiveness  of  the  Korean  NDI, 
the  researcher  reviews  the  characteristics  and  problems  for  various  existing  models  and  the 
competitiveness  analysis  results  for  other  civil  industries.  He  then  compares  the  attributes  of 
the  NDI  with  civil  industries  and  analyzes  the  critical  technology  competitiveness  of  the  NDI 
for  major  nations.  In  addition,  he  examines  the  defense  policies  under  the  worldwide  reform 
of  the  defense  management  environment  and  the  global  change  of  the  international  defense 
market. 


The  researcher  develops  the  Pentagon-Defense  8  Factors  (P-D8F)  model,  a 
proposed  optimum  analysis  model  for  the  NDI,  by  applying  the  results  of  the  analysis  for 
various  existing  models  while  considering  the  unique  characteristics  of  the  NDI.  He  then 
analyzes  the  international  competitiveness  of  the  Korean  NDI  by  using  the  P-D8F  model, 
and  derives  the  reinforcement  strategy  to  enhance  the  global  competitiveness  for  the 
Korean  NDI.  Finally,  he  suggests  a  possible  defense  acquisition  policy  to  promote  the 
desired  technological  innovation  and  to  improve  the  management  environment  effectively. 
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Introduction 

Since  the  cessation  of  the  Cold  War,  most  nations  have  focused  their  energies  on 
enhancing  their  economic  and  technological  powers  instead  of  on  reinforcing  their  military 
power.  In  particular,  the  powerful  military  nations — such  as  the  US,  France  and  Russia — 
steadily  carry  out  various  projects,  transforming  military  companies  into  civil  companies. 
They  have  also  effectively  accomplished  the  military  transformation  based  on  the  Effect- 
based  Operation  (EBO)  to  transform  the  defense  industrial  base  (DoD,  2003). 

With  the  rapid  transition  of  the  global  defense  management  environment,  along  with 
the  reinforcement  of  the  WTO  system  and  the  increase  of  global  competition,  the  NDI  can't 
be  maintained  by  the  long-sustained  government  control  and  support  anymore.  Many 
defense  companies  try  to  make  a  good  profit  and  survive  in  the  global  environment.  For  the 
past  40  years,  the  government  in  Korea  has  played  a  leading  role  in  producing  and 
supplying  military  materiel.  The  NDI  has  been  promoted  strongly  by  government  initiatives, 
but  the  most  advanced  military  materiel  hasn't  been  researched  or  developed  with  domestic 
technology,  capabilities  and  resources.  The  critical  military  technologies  haven't  been 
acquired  yet,  and  the  international  competitiveness  of  the  Korean  NDI  is  very  low  in 
comparison  with  that  of  developed  countries.  Therefore,  Korean  companies  don't  have  the 
capabilities  to  research  and  develop  the  most  advanced  weapons  to  be  competitive 
internationally  (Lee,  2007). 

For  these  reasons,  a  great  deal  of  extensive  research  must  be  conducted  in  an  effort 
to  discover  the  proper  strategy  to  strengthen  the  international  competitiveness  of  the  NDI. 
However,  due  to  the  lack  of  a  competitiveness  analysis  model,  there  haven't  been  any 
considerable  efforts  to  research  or  to  develop  the  systematic  improvement  methodologies  to 
solve  the  above-mentioned  problems.  To  overcome  these  problems  effectively,  an  original 
analysis  model,  which  fully  takes  into  account  the  defense  characteristics  based  on  the  civil 
industries  model,  is  required.  Thus,  the  researcher  has  developed  an  innovative  analytic 
strategy  for  promoting  the  international  competitiveness  of  the  NDI. 

The  purposes  of  this  study  are  as  follows.  The  first  purpose  is  to  develop  and 
validate  the  optimum  analysis  model  for  the  Korean  NDI,  based  on  the  analytic  results  for 
various  existing  models  and  the  consideration  of  the  NDI’s  unique  characteristics.  The 
second  purpose  is  to  analyze  and  review  the  international  competitiveness  of  the  Korean 
NDI  and  to  compare  it  with  the  competitiveness  of  the  NDIs  in  other  major  nations  by 
applying  the  proposed  model.  The  final  objective  is  to  suggest  the  reinforcement  strategy 
that  may  enhance  the  global  competitiveness  of  the  Korean  NDI. 

Development  of  International  Competitiveness  Analysis  Model 
for  the  Korean  National  Defense  Industry 

1.  Characteristics  of  the  Korean  National  Defense  Industry 

The  Korean  NDI  has  several  different  characteristics  than  do  other  civil  industries  or 
the  NDIs  of  developed  countries.  First,  the  NDI  is  a  government-dependant  industry 
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contributing  to  national  security  and  showing  monopolistic  and  oligopolistic  characteristics 
under  government  control  and  support.  Second,  the  NDI  is  an  industry  that  contributes  and 
enhances  the  national  military  power  and  fosters  the  national  economy.  Therefore,  its 
economical  efficiency  isn’t  considered,  and  one-sided  investment  is  mandated  by 
government  policy.  As  a  result  of  this  policy,  high  national  economic  growth  is  unintentionally 
achieved  in  the  confined  fields  of  the  shipbuilding,  aviation,  and  automobile  industries.  Third, 
the  military  authorities  require  the  military  materiel  with  the  best  quality  rather  than  at  the 
lowest  cost.  And  the  government  controls  the  demand  and  supply  within  the  scope  of  the 
defense  budget  and  the  quantity  of  each  service’s  request.  The  law  of  supply  and  demand 
does  not  work  in  the  NDI.  There  is  always  a  balance  of  supply  and  demand  in  the  market. 
Finally,  excessive  plant  investment  and  the  long-term  diversion  of  capital  from  the  defense 
industry  are  required.  Thus,  the  operating  rate  is  very  low,  and  the  quantity  is  unstable  and 
limited  depending  upon  the  government  policy  and  the  surrounding  environment.  It's  hard  to 
estimate  demand  for  just  5-plus  years. 

2.  Korean  Acquisition  Process  of  Defense  Materiel 

In  2006,  the  restructuring  of  the  entire  acquisition  system  in  Korea  was  completed. 
One  of  the  goals  of  this  was  to  redirect  the  government  acquisition  policy  to  Defense  Reform 
2020  (MND,  2005),  stressing  self-reliant  defense  based  on  the  Korea-US  alliance.  Other 
goals  were  to  pursue  the  transformation  for  Technological  Forces  and  to  effectively  increase 
the  investment  in  the  defense  R&D  (research  and  development)  budget  (6.5%  average). 

The  guidelines  for  this  key  acquisition  reform  were  to:  require  transparency,  increase 
efficiency,  secure  expertise  and  professionals,  enhance  competitiveness,  and  strengthen 
international  cooperation. 

Acquisition  flow  has  been  reorganized,  as  shown  in  Figure  1.  Formerly,  requirement 
and  procurement  were  driven  by  the  services  (JCS,  Army,  Navy,  and  Air  Force).  The 
acquisition  organizations  such  as  MND  (Ministry  of  national  Defense),  DPA  (Defense 
Procurement  Agency),  PMOs  (Army-Navy-Air  Force  Project  Management  Office),  T&EO 
(Test  and  Evaluation  Office),  DQAA  (Defense  Quality  Assurance  Agency),  and  ADD 
(Agency  for  Defense  Development)  weren’t  well  connected  organically.  The  PPBEES 
(planning,  programming,  budgeting,  execution,  and  evaluation  system)  did  not  work 
continuously  and  systematically  during  the  lifecycle  of  materiel  acquisition.  The  ADD,  along 
with  other  organizations,  used  to  be  at  the  center  of  acquisition.  Presently,  requirements 
come  from  services.  But  the  DAPA  (Defense  Acquisition  Program  Administration) — into 
which  8  organizations  related  to  defense  acquisition  have  merged — is  now  at  the  center  of 
acquisition,  while  the  ADD  has  remained  at  the  center  of  R&D  activities. 

Figure  2  shows  the  acquisition  process — including  the  system  development  flow, 
which  is  somewhat  similar  to  that  in  the  US.  Requirements  come  from  the  services  and  the 
MND.  After  preliminary  studies,  the  DAPA  decides  the  mode  of  development.  Figure  2  also 
shows  the  flow  of  system  development  from  S&T  to  deployment.  Preliminary  studies  are 
dedicated  to  concept  development,  alternative  analysis  and  interoperability  analysis  to 
establish  acquisition  trade-offs.  The  exploratory  development  phase  is  composed  of  concept 
exploration  and  technology  development  stages;  engineering  efforts  are  dedicated  to 
authorizing  the  operational  concepts  and  required  capabilities  and  to  eliminating  technical 
risks  before  a  program  enters  the  system  development  phase.  The  system  development 
phase  is  composed  of  system  integration  and  system  demonstration  stages.  Constructive 
and  virtual  simulation  models  are  utilized  for  design  and  system  verification.  System  and 
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subsystem  performance  analysis  tools  are  used  for  verifying  that  designs  are  compliant  with 
the  requirements  (DAPA,  2007). 

The  DAPA  organization  is  shown  in  Figure  3.  The  Commissioner  (at  the  Vice 
Minister  level)  is  the  head  of  DAPA.  At  the  next  lower  level,  the  Vice  Commissioner 
oversees  Acquisition  Planning,  Defense  Industry  Promotion,  Analysis/T&E,  and  Policy  & 
Public  Relation  Management  Bureaus.  There  is  also  the  Program  Management  Agency, 
which  controls  several  programs  with  the  help  of  IPTs. 


Requirement  &  Acquisition  Requirement  Acquisition 

Procurement 

[Former]  [Present] 


Figure  1.  Key  Thrusts  for  Acquisition  Reform  (Reorganization) 
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Development  Mode  Decision 
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Figure  2.  Acquisition  Process,  Including  System  Development  Flow 
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Figure  3.  Organization  of  Defense  Acquisition  Program  Administration  (DAPA) 

3.  International  Competitiveness  Analysis  Model  for  the  Korean  National 
Defense  Industry 

The  existing  civil  competitiveness  models  (Porter,  1998)  are  insufficient  for  analyzing 
the  Korean  NDI  because  of  its  different  characteristics.  The  rule  of  market  economy  doesn't 
work,  as  the  operations  of  most  firms  are  under  the  government’s  control.  In  the  NDI,  the 
economy  system  is  governed  rather  by  the  government-initiated  economy  than  the  private- 
initiated  economy.  The  sole  consumer  is  the  government,  and  the  NDI’s  supply  and  the 
government’s  demand  always  meet  in  the  market.  Government  programs,  instead  of  the 
mechanisms  of  market  economy,  decide  demand. 

As  for  Korea,  all  competitiveness  factors  in  the  NDI  are  dictated  by  the  government's 
intention  and  policy.  The  foreign  policy  and  international  relations  of  the  major  powerful 
nations,  such  as  the  US,  China,  and  other  surrounding  nations,  are  also  important  factors 
with  which  to  analyze  the  competitiveness  of  the  NDI.  And  chance  is  important  because  it 
creates  discontinuities  and  plays  its  role  partly  by  altering  conditions  in  the  competitiveness 
models.  These  characteristics  must  be  considered  as  the  optimized  models  are  built  (Lee, 
2000). 


By  applying  the  analysis  results  for  various  existing  models  and  considering  the 
unique  characteristics  of  the  Korean  NDI,  the  researcher  was  able  to  develop  the  optimum 
analysis  model  for  the  Korean  NDI.  In  case  of  the  Korean  NDI,  5  determinants  to  achieve 
the  national  competitive  advantages  among  the  nations  exist:  factor  conditions,  the  strategy 
and  rivalry  among  the  firms,  related  and  supporting  industries,  demand  conditions,  and  the 
government  defense  policy.  And  3  influencers  of  true  competitiveness  are:  the  industrial 
cluster,  the  defense  policy  and  the  relations  of  foreign  nations,  and  chance. 

Figure  4  shows  the  Pentagon-Defense  8  Factors  Model  (P-D8F) — with  5 
determinants  and  3  influencers  for  competitiveness  of  the  Korean  NDI.  The  factors  of  the 
national  competitive  advantage  according  to  determinants  and  influencers  are  shown  in 
Table  1. 
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Figure  4.  The  Pentagon-Defense  8  Factors  Model  for  the  Korean  NDI 


Table  1.  Determinants  and  Influencers  of  Pentagon-Defense  8  Factors  Model  for  the 

Korean  NDI 


Determinants  and 
Influencers 

Factors  for  National  Competitiveness  Advantage 

Factor 

Conditions 

■  Human,  physical,  knowledge,  and  capital  resources 

■  Infrastructure 

■  Mechanisms  creating  competitive  advantage 

Demand 

Conditions 

■  Demand  size  and  pattern  of  growth 

■  Internationalization  of  domestic  demand 
•  Home  demand  composition 

Deter¬ 

minants 

Related  and 

Supporting 

Industries 

■  Presence  of  internationally  competitive  supplier 
and  related  industry 

•  Competitive  advantage  of  supplier  and  related  industry 

Firms  Strategy 
and  Rivalry 

■  Management  strategy  and  structure  of  domestic  firms 

■  Vision,  goals,  and  leadership 

■  Rivalry  among  existing  competitors  and  threat  of  a  new  entrant 

Government 

Defense 

Policy 

■  Policies  toward  defense  acquisition  and  capital  market 

•  Product  standard  and  regulation 

•  Capital  market  regulation,  tax  policy,  and  antitrust  law 

Influen¬ 

cers 

Industrial 

Cluster 

■  Presence  of  industrial  park  and  complex,  high-technology  park, 
and  evolutional  process  of  cluster 

■  Existence  of  cluster  for  several  industries 
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Defense 

Policy  and 
Relation 
(Foreign 
Nation) 

■  Political  decision  by  foreign  government 

■  Variations  of  international  defense  environment 
and  military  expenditure 

•  Regulations  of  arms  export  and  technology  transfer 

Chance 

■  Invention,  technology  innovation,  and  oil  shock 

■  Significant  shift  in  world  financial  market 

■  International  dispute  and  regional  war 

International  Competitiveness  Analysis  of  the  Korean  National 
Defense  Industry  Using  the  Pentagon-Defense  8  Factors  Model 

1.  Factor  Conditions 

In  2007,  the  Korean  NDI  was  composed  of  88  companies  producing  diverse  systems 
and  components  of  10  fields  of  defense  materiel.  Yet,  they  show  very  low  operating  rates  of 
50%  to  60%,  low  profit  margins  of  8.1%,  and  very  low  revenue  per  capita  of  $240  million 
(KDIA,  2007).  All  data  hardly  come  up  to  those  of  Korean  civil  industries  and  foreign 
countries.  The  international  competitiveness  of  the  Korean  NDI  in  the  world  market  is  very 
weak.  The  continuous-growth  strategy  must  be  propelled  by  defense  reform  and  self-reliant 
cooperation. 


Table  2.  Variations  of  Operating  Rate  for  the  Korean  NDI 


Year 

1990 

1992 

1994 

1996 

1998 

2002 

2003 

2004 

2005 

2006 

Operating  Rate 
(%) 

58.2 

59.8 

56.1 

55.7 

57.4 

54.5 

57.3 

56.1 

57.8 

60.6 

Table  3.  Productivity  &  Technology  Capability  for  the  Korean  NDI  in  2005 


Item 

Return  On  Equity 
(%) 

Revenue  Per 
Capita  ($M) 

Operating  Rate 
(%) 

Tech.  Level 

(%) 

Figures 

8.1 

240 

57.8 

67 

Remarks 
(civil  industries) 

12.2 

500 

79.8 

80 

Table  4  shows  the  variations  of  full-time  employees  of  the  Korean  NDI.  Since  2000, 
the  number  of  total  employees  of  the  NDI  has  been  decreasing  gradually  due  to  factory 
automation  and  reduction  in  demand  for  conventional  weapons.  Furthermore,  expert 
research  engineers  are  no  more  than  2,000,  10%  of  full-time  NDI  employees.  In  addition,  the 
number  of  Korean  defense  R&D  employees  in  the  government — including  the  associated 
Institutes  and  Agency — is  just  4,000,  quite  small  compared  to  200,000  of  the  US,  25,000  of 
Germany,  and  18,000  of  Taiwan.  Lacking  expert  manpower,  the  technical  level  of  the  NDI 
can’t  improve  and  is  still  far  from  that  of  developed  countries. 
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Table  4.  Variations  of  Full-time  Employees  for  the  Korean  NDI 


Year 

1988 

1991 

1996 

2002 

2003 

2004 

2005 

2006 

Number 

23,418 

27,794 

29,640 

23,179 

23,184 

23,007 

22,226 

20,912 

According  to  the  US  Military  Critical  Technologies  List  ( MTCL ),  which  shows  country- 
by-country  estimates  of  the  general  status  of  technological  capabilities,  Korea  doesn't  retain 
the  critical  technologies  of  Aeronautics  Systems,  Energy  Systems,  Sensor  and  Laser,  and 
Space  Systems.  One  the  other  hand,  Korea’s  technology  levels  of  Information  Systems, 
Materials,  and  Nuclear  Systems  come  more  or  less  close  to  those  of  the  developed 
countries  (DoD,  2004).  In  this  regard,  the  Korean  government  tries  to  enhance  its  defense 
technology  competitiveness  through  technology  interactions  (spin-off  and  spin-on)  with 
commercial  sectors,  and  lay-out  schemes  fostering  the  high  technology  areas  strategically. 

The  defense  R&D  budget  is  about  $1 ,060  million.  That  was  4.7%  of  the  defense 
budget  of  $22,513  million  and  14.1%  of  the  defense  materiel  improvement  budget  of  $7,499 
million  in  2006  (MND,  2007).  It  is  gradually  increasing  every  year,  but  the  defense  R&D 
budget  in  2005  ($740  million)  was  quite  small  compared  to  major  nations — $82,250  million  in 
the  US,  $4,690  million  in  the  UK,  and  $4,850  million  in  France  (SIPRI,  2006).  The  Defense 
budget  in  2006  national  GDP  is  still  low;  just  2.6%  compared  with  the  US,  3.7%,  China, 

3.9%,  and  Russia,  4.9%  (Hackett,  2007).  The  defense  budget  versus  the  GDP  and  the 
defense  R&D  budget  versus  thte  defense  budget  have  to  be  increased  gradually  up  to 
developed  countries’  levels. 


Table  5.  Variations  of  Defense  Budget  and  Defense  R&D  Budget 


Year 

1990 

1992 

1996 

1998 

2002 

2003 

2004 

2005 

2006 

Defense 
Budget  ($M) 

6,608 

8,309 

12,243 

13,800 

16,364 

17,515 

18,941 

20,823 

22,513 

Defense  R&D 
Budget  ($M) 

143 

235 

374 

479 

723 

739 

797 

929 

1,060 

Rate  (%) 

2.1 

2.8 

3.1 

3.5 

4.4 

4.2 

4.2 

4.5 

4.7 

Table  6.  Defense  R&D  Budget  for  Major  Nations  (2005) 


(at  constant  2003  prices 


Country 

US 

UK 

France 

Italy 

Japan 

Russia 

Korea 

Defense  Budget 
($100M) 

4,782 

483 

462 

272 

421 

210 

162 

Defense  R&D 
Budget  ($100M) 

822.5 

46.9 

48.5 

6.8 

15.2 

23.7 

7.4 

Rate  (%) 

17.2 

9.7 

10.5 

2.5 

3.6 

11.3 

4.5 
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2.  Demand  Conditions 

Demand  is  decided  within  the  scope  of  requirements  of  the  services  and  the  defense 
budget.  It  is  difficult  for  the  NDI  to  estimate  the  demand  for  even  5  years.  The  materiel 
improvement  programs  of  the  Armed  Forces  are  established  by  the  MND  and  the  DAPA 
through  a  5-year,  mid-term  plan.  However,  the  procurement  budget  is  finally  set  with  the 
adjustment  and  deliberation  of  the  government  (Ministry  of  Strategy  and  Finance)  and  the 
National  Assembly  annually.  Therefore,  it  quite  restricts  the  NDI  from  establishing  a  mid- 
and  long-term  management  plan. 

Recently,  Korean  domestic  demand  for  defense  materiel  has  been  limited  because 
services  want  the  cutting-edge  weapon  systems,  yet  the  NDI  lacks  critical  technologies. 
Industry  promotion  is  restricted  due  to  the  insufficient  domestic  demand,  and  the  creation  of 
a  new  market  is  ineffective.  The  core  technology  level  compared  to  that  in  leading  countries 
is  67%  in  2004;  most  areas  of  cutting-edge  technology  are  even  more  vulnerable. 

Table  7  shows  the  variations  of  gross  sales  of  the  Korean  NDI.  Total  sales  in  2006 
reached  $5,452  million — 45  times  greater  than  the  total  sales  in  1988,  $120  million.  But,  the 
growth  rate  of  total  sales  (2.5%)  is  very  low  compared  with  the  6.3%  of  civil  industries  in 
2006.  The  operating  income  increases  to  $267  million — 4  times  greater  than  the  $61  million 
in  1988.  On  the  other  hand,  the  operating  profit  margin  of  4.9%  reaches  5.3%  of  civil 
industries,  while  the  ordinary  profit  margin  of  3.0%  is  lower  than  5.7%  of  civil  industries. 


Table  7.  Variations  of  Gross  Sales  for  the  Korean  NDI 


Year 

2001 

2002 

2003 

2004 

2005 

2006 

Total  Sales  ($M) 

3,705 

4,366 

4,269 

4,644 

5,317 

5,452 

Growth  Rate  (%) 

11.1 

17.8 

-2.3 

8.8 

14.5 

2.5 

Operating  Income  ($M) 

221 

151 

154 

141 

250 

267 

Ordinary  Income  ($M) 

-115 

25 

56 

86 

149 

163 

The  export  of  defense  materiel  by  the  NDI  alone  is  difficult.  To  make  it  possible,  the 
cooperation  of  the  government,  military,  and  the  NDI  is  required.  As  for  the  exports  of 
defense  materiel,  the  training  and  education  programs  and  logistics  are  transferred  to  the 
purchasing  country.  The  establishment  of  the  export  marketing  strategy  by  the  NDI  is 
restricted  because  customers  are  foreign  governments. 


The  number  of  major  conventional  weapons  being  exported  and  imported  is 
decreasing  steadily,  whereas  the  demand  for  advanced  materiel  systems  is  increasing 
gradually.  In  2006,  Korea’s  arms  imports  amounted  to  about  $600  million  (the  9th  largest  in 
the  world  market),  while  Korean  arms  exports  were  numbered  at  about  $255  (the  20th 
largest,  taking  just  0.2%  in  the  world  market)  (KDIA,  2007).  In  the  period  2001  through  2005, 
Korean  arms  exports  ranked  the  17th  largest  ($337  million),  and  arms  import  ranked  the  9th 
largest  ($2,561  million)  (SIPRI,  2006).  Thus,  this  severe  imbalance  of  the  trade  leads  to  the 
shrinking  of  the  Korean  NDI.  Domestic  development  of  advanced  weapons  must  increase  if 
the  defense  economy  is  to  improve. 
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Table  8.  Variations  of  Arms  Export  Sales  for  the  Korean  NDI 


Year 

1991 

1995 

1999 

2001 

2002 

2003 

2004 

2005 

2006 

Export  Sales  ($M) 

90.6 

77.0 

196.6 

237.2 

143.9 

240.6 

417.8 

262.3 

255.2 

3.  Related  and  Supporting  Industries 

The  Korean  NDI  is  composed  of  88  main  companies,  producing  the  373  kinds  of 
defense  materiel.  And  20  to  100  subcontractors  work  with  these  main  contractors.  There  are 
also  some  defense-related  organizations  and  civil-related  industries  that  interconnect  to 
develop  and  produce  the  defense  materiel.  The  Korean  NDI  is  classified  into  two  structures 
with  vertically  and  horizontally  specialized  relations.  The  pyramidal  configuration  of  the 
Korean  NDI  is  constructed  with  the  vertical  relations  among  main  contractors, 
subcontractors,  and  components  suppliers,  and  the  horizontal  relations  among  weapon 
industries  producing  particular  weapons  such  as  fire  power,  automobiles,  aeronautics, 
warship,  and  missile  systems,  etc. 

Figure  5  shows  the  pyramidal  configurations  of  related  and  supporting  industries  for 
the  Korean  NDI.  Generally,  the  vertical  and  horizontal  structure  of  pyramidal  configurations 
is  the  best  solution  to  optimizing  the  effectiveness  of  the  NDI.  That  is,  when  the  lower 
members  (such  as  material  and  component  suppliers)  are  strong  and  solid,  raw  materials 
and  components  are  supplied  quickly  and  inexpensively.  This  structure  has  the  advantage  of 
maximizing  technology  development  through  competitive  activities.  Furthermore, 
interactions  and  exchanges  among  weapon  systems  industries  in  the  horizontal  structure 
can  foster  the  competition  of  other  weapon  systems  industries;  likewise,  interaction  with  civil 
industries  can  bring  out  critical  technology  innovation  in  civil  industries. 
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Figure  5.  Pyramidal  Configuration  of  Related  and  Supporting  Industries 

for  the  Korean  NDI 
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4.  Firms  Strategy  and  Rivalry 

Most  Korean  defense  firms  lack  competitiveness  due  to  the  monopolistic  and 
oligopolistic  systems  of  the  Korean  NDI.  Most  defense  acquisition  programs  are  still 
classified,  and  much  information  is  still  limited  to  the  public.  The  firms  have  difficulty 
establishing  mid-  and  long-term  management  plans  due  to  this  limited  access  to  detailed 
information.  The  entry  barrier  of  newcomers  is  very  high,  and  most  firms  make  few  efforts  to 
develop  the  advanced  critical  technologies. 

Defense  revenue  was  only  7.8%  of  the  defense  industry’s  total  sales  in  2005,  and 
the  low  productivity  of  most  firms  impedes  the  progress  of  the  defense  industry.  Firms  are 
faced  with  excessive  financial  cost  due  to  their  lack  of  budget  compared  to  initial  investment 
in  large-scale  R&D  projects.  Every  year,  the  investment  of  firms  decreases — by  $7.6  million 
in  2000,  $5.1  million  in  2002,  and  $4.5  million  in  2004. 

In  this  environment,  the  aggressive  management  strategy  of  industries  isn't 
established  enough.  Entrepreneurs  and  employers  don't  formulate  great  visions  and  goals 
for  their  companies  because  there  is  almost  no  rivalry  among  existing  competitors  or  threat 
of  new  entrants. 

5.  Government  Defense  Policy 

Despite  numerous  attempts  to  improve  the  Korean  acquisition  system,  the  issues  of 
inefficiency  and  noncompetitiveness  within  the  defense  industry  are  not  yet  resolved.  In 
recent  years,  the  government  has  tried  to  initiate  drastic  improvement  in  defense  acquisition 
policy.  The  domestic  R&D  of  advanced  military  materiel  takes  precedence  over  overseas 
acquisition.  In  addition,  the  government  has  tried  to  establish  acquisition  plans  that  ensure 
the  balanced  development  of  advanced  military  materiel,  in  accordance  with  the 
requirements  of  the  Joint  Military  Strategy  and  within  the  limits  of  the  national  budget. 

The  government  has  also  discontinued  the  current  specialization  and  grouping 
system  of  the  defense  companies  to  improve  industry  competitiveness.  Thus,  technology 
innovations  among  associated  companies  will  be  induced.  And  the  technology  level  of 
defense  companies  can  be  raised  up  to  the  level  of  the  developed  countries. 

The  government  has  endeavored  to  work  in  close  cooperation  with  defense 
companies  to  enhance  exports,  and  has  reinforced  the  export  administration  to  support 
them.  To  help  establish  vision  and  goals  within  defense  companies,  the  government  has 
expanded  its  disclosure  of  information  on  defense  investment  projects  to  the  public. 

6.  Industrial  Cluster 

National  competitive  advantage  is  generally  more  remarkable  in  industrial  clusters 
than  in  individual  industries.  Industrial  clusters  often  emerge  and  begin  to  grow  naturally, 
such  as  Silicon  Valley  in  the  US.  But,  in  Korea,  governmental  policy  initiated  many  industrial 
clusters,  such  as  industrial  parks,  complexes,  and  high-technology  parks  in  several  areas. 
The  government  has  established  some  special  industrial  and  R&D  regions  by  providing 
specialized  infrastructures  and  technical  centers. 

Figure  6  shows  one  of  industrial  clusters  of  the  Korean  NDI  in  Changwon  City. 
Related  and  supporting  industries,  agencies,  institutes,  academia,  research  centers,  society 
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and  associations,  and  suppliers  are  interconnected  to  research,  develop,  produce,  and 
operate  the  defense  materiel.  Another  type  of  cluster,  the  defense  R&D  industrial  cluster  in 
Daejeon  City  and  Chungcheong  Province,  is  shown  in  Figure  7.  It  is  a  unique  one  in  Korea 
in  the  respect  that  most  institutes,  agencies,  and  universities  related  with  R&D — in  military, 
government,  and  civil  fields — are  centralized  in  this  region.  Nevertheless,  the  Industrial 
Clusters  are  not  yet  complete,  as  they  are  not  creating  the  necessary  high  value-added 
products  due  to  the  lack  of  the  proper  establishment  of  knowledge-based  networks. 


Shipbuilding  Ind. 


Aero.  Ind. 

Comm.  Ind. 

Related  Industries 

Elec.  Ind. 

Semicon.  Ind. 

Precision 
Machinery  Ind. 


Society  & 
Association 

71 


Supp 

orting 

l _ 

Education  & 
Research  & 
T&E  Agency 

Industries 

i 

Facility  & 
Machinery 
Suppliers 


Raw  &  Mid 
Material 
Suppliers 

Part  & 
Component 
Suppliers 


— 


Figure  6.  Defense  Industrial  Cluster  for  the  Korean  NDI  in  Changwon  City 
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Figure  7.  Defense  R&D  Industrial  Cluster  in  Daejeon  City  and  Chungcheong  Province 


ACQUISITION  RESEARCH:  CREATING  SYNERGY  FOR  INFORMED  CHANGE  -  399  - 


7.  Chance 

Chances  unexpectedly  appear  that  have  little  to  do  with  specific  circumstances  in  a 
nation.  They  are  important  because  they  incur  discontinuities  that  allow  shifts  in  the 
competitive  position.  In  the  history  of  modern  Korean  economy,  chances  that  have  been 
particularly  important  in  influencing  competitive  advantage  are  as  follows:  technology 
innovation,  oil  shocks,  world  financial  crises,  high  volatility  of  exchange  rate,  international 
disputes  and  regional  wars.  Chances  play  a  role  partly  by  altering  the  determinants  in  the 
Pentagon-Defense  8  Factors  Model. 

Korea,  like  other  nations,  is  facing  significant  changes  in  its  defense  outlook.  Since 
the  2000  Joint  Declaration  (Pyeongyang  in  North  Korea,  June  15),  South  Korea  and  North 
Korea  have  promised  proactive  exchanges  and  cooperation.  However,  the  Cold  War 
structure  is  still  lingering,  with  factors  of  potential  regional  friction  (such  as  the  North  Korean 
nuclear  issue).  Countries  in  Northeast  Asia  are  engaged  in  constant  competition  to  expand 
their  influence  in  this  region  while  continuing  to  make  efforts  to  modernize  and  transform 
their  military  forces. 

In  recent  years,  as  a  result  of  dramatic  development  in  science  and  technology,  the 
Korean  defense  companies  have  been  subject  to  sweeping  changes.  There  has  been  also  a 
significant  qualitative  change  in  the  nature  of  technology  because  civil  technology  has 
become  increasingly  important  for  weapon  systems.  The  technological  cooperation  of  the 
Korean  NDI  with  civil  industries  is  underway,  and  it  presents  the  NDI  with  a  chance  to 
enhance  its  technological  competitiveness.  The  oil  shocks  greatly  reduce  the  defense 
budget  in  energy-dependent  nations  such  as  Korea,  and  downscale  the  size  of  the 
worldwide  defense  market.  However,  the  US’s  Global  War  on  Terror  and  the  local  wars  in 
the  Middle  East  and  Asia  have  shown  the  sudden  increase  in  demand. 

8.  Defense  Policy  and  Relation  (Foreign  Government) 

During  the  post-Cold  War  period,  developed  countries  have  gained  military 
superiority  by  protecting  their  national  critical  technological  capabilities.  They  have 
significantly  increased  their  defense  R&D  budget  to  develop  new  technologies 
independently,  and  have  reinforced  their  control  over  and  prohibit  the  divulgement  of  critical 
military  technologies  and  products  to  foreign  countries.  In  addition,  they  have  expanded  the 
development  of  dual-use  technology,  a  spin-on  of  civil  technology  and  spin-off  of  military 
technology. 

Most  nations  control  and  monitor  the  transfer  of  sensitive  technologies  to  other 
nations  while  improving  thte  export-control  processes  to  promote  and  protect  their  domestic 
defense  companies.  With  armaments  cooperation  programs  to  strengthen  their  military  and 
industrial  relationships,  they  bind  other  nations  as  their  partners  in  strong  security  coalitions. 
In  recent  years,  the  restrictions  of  arms  export  and  military  technology  transfer  to  competing 
countries  are  much  more  stringent. 

To  cope  with  the  international  defense  environment,  Korea  has  increased  military 
expenditures  to  develop  critical  technologies  independently,  and  has  participated  in 
cooperative  programs  with  foreign  advanced  companies  to  introduce  and  co-develop 
advanced  technology  and  to  foster  its  arms  exports.  Korean  has  even  tried  to  increase  its 
arms  exports  in  order  to  improve  demand  conditions  within  the  NDI. 
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Defense  Acquisition  Policy  and  Defense  Industrial  Base 
Reinforcement  Strategy 

1.  Enlargement  of  Defense  Budget 

Korea’s  defense  budget  in  its  GDP  is  still  low  compared  to  other  countries.  In  2006, 
the  ratio  of  the  defense  budget  to  the  GDP  was  just  2.6%,  obviously  quite  low  compared  to 
other  nations’  ratios:  3.7%  of  US,  3.9%  of  China,  and  4.9%  of  Russia.  Most  nations  engaged 
in  inner-  and  outer-conflicts  usually  allocate  an  average  of  5-6%  of  their  GDP  to  the  defense 
budget.  The  annual  growth  rate  of  Korea’s  defense  budget  is  expected  to  increase  up  to 
about  9%  through  2015,  making  the  budget  share  out  of  GDP  go  up  gradually  to  3%  in  2015 
(MND,  2005).  However,  the  budget  share  has  to  increase  gradually  up  to  the  level  of 
developed  countries. 

In  addition,  R&D  financial  resources  must  be  expanded  in  order  to  support  the 
acquisition  policy.  The  defense  R&D  budget  has  to  increase  to  10%  of  the  defense  budget 
to  enhance  the  international  competitiveness  of  the  NDI  in  201 5  from  just  4.7%  in  2007.  The 
dual-use  technology  programs  are  reinforced  to  share  effectively  the  limited  defense  R&D 
budget  with  the  civil  sectors. 

2.  Reinforcement  of  Factor  Conditions 

In  planning  its  acquisition  strategy,  Korea  must  give  the  utmost  priority  to  domestic 
R&D  of  Arms  if  it  is  to  both  enhance  the  critical  technology  capabilities  of  the  NDI  to  the  level 
of  developed  countries,  and  to  raise  the  operating  rate  of  the  NDI  to  the  level  of  civil 
industries.  The  NDI  must  participate  more  in  R&D  if  it  is  to  lead  advanced  arms  development 
and  to  enlarge  its  technological  capabilities.  The  ADD  also  has  to  be  transformed  into  a  core 
technology  and  system-of-systems-centric  institute,  and  the  NDI's  role  in  developing  general 
weapon  systems  has  to  be  expanded. 

As  mentioned  previously,  the  number  of  expert  research  engineers  within  the  NDI  is 
just  about  2,000.  Furthermore,  the  number  of  Korean  defense  R&D  manpower  under  the 
government  side  is  just  4,000,  which  is  far  less  than  the  200,000  in  the  US.  Due  to 
insufficient  expert  manpower,  the  technical  level  of  the  NDI  cannot  be  as  good  as  other 
developed  countries.  To  ensure  comparative  advantages  over  developed  countries,  more 
expert  researchers  and  engineers  are  necessary. 

3.  Innovation  of  Defense  Technology 

In  the  US,  revolutionary  innovations  in  military  technology  traditionally  come  from 
subcontractors  or  niche  firms,  and  these  firms  frequently  go  on  to  dominate  the  markets. 
These  monumental  leaps  are  infrequently  developed  by  the  prime  firms  of  their  time.  This 
paradigm  has  been  observed  frequently  in  many  other  industries  as  well. 

In  Korea,  the  ADD  has  taken  the  lead  in  developing  innovative  defense  technology. 
But,  it  is  difficult  for  the  ADD  to  play  a  leading  role  anymore.  The  ADD  has  to  cooperate  with 
academia,  institutes,  research  centers,  and  industries,  and  to  transfer  acquired  technologies 
and  support  them  steadily  to  innovate  defense  technologies.  If  necessary,  the  government 
has  to  change  the  acquisition  strategy  to  ensure  a  competition-driven  innovation.  Also,  the 
government  must  stimulate  competitions  through  incentives,  and  designate  multiple  sources 
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to  produce  the  desired  technology.  Probably  the  most  important  thing  for  improving  and 
developing  innovative  technologies  is  to  ensure  robust  funding. 

4.  Establishment  of  Competitive  Environment 

The  Specialization  and  Affiliation  System  that  the  companies  are  designated  to 
participate  in  the  acquisition  (R&D  or  purchase)  programs  was  introduced  to  stimulate 
technology  development  and  to  save  financial  resources  by  preventing  excessive 
competition.  On  account  of  the  excessive  preferential  authority  of  designated  companies, 
strong  barriers  are  formed  that  block  new  entries  and  obstruct  technological  innovation. 

To  enhance  the  NID’s  industrial  competitiveness  and  to  induce  technological 
innovation,  the  above  system  is  expected  to  be  abolished  by  the  end  of  2008.  On  the  other 
hand,  in  order  to  promote  technological  development  and  efficiency,  Korean  decision¬ 
makers  must  prepare  some  criteria  and  processes  to  prevent  excessive  competition  and  to 
institutionally  ensure  the  entrances  of  small-  and  medium-sized  companies. 

In  addition,  reasonable  designation  and  management  of  defense  materiel  and 
companies  have  to  be  established;  likewise,  to  promote  competition  efficiency,  appropriate 
criteria  for  timing,  scope  and  items  in  designating  the  defense  materiel  have  to  be  set.  The 
government  has  to  prevent  inefficiency  by  reviewing  requirements  periodically,  and  must 
consider  appropriate  measures  to  designate  and  manage  the  defense  materiel  and 
companies  for  the  competitive  system. 

5.  Improvement  of  Management  Condition 

The  government  should  cover  some  key  expenses  to  help  the  companies  reform 
themselves  and  actively  carry  out  additional  activities.  The  supporting  funds  to  promote  the 
competitiveness  of  the  NDI  have  to  be  secured,  and  enough  subsidies  to  develop  critical 
technologies  and  construct  infrastructures  of  the  NDI  have  to  be  granted.  The  cost-based- 
contract  system  of  defense  materiel  should  be  improved  to  enhance  competitiveness  for 
exports  by  reflecting  more  expenses. 

The  government  has  to  establish  mid-  and  long-term  strategies  to  reinforce  the 
industrial  base  through  regular  investigation;  it  must  also  induce  the  rational  supply  system 
and  promote  the  efficiency  of  resources  through  continuous  analysis  of  the  defense 
industry's  demand  and  supply  status.  The  government  also  has  to  expand  industrial  stratum 
by  fostering  defense-specialized  small  and  medium-sized  business.  Small  and  medium¬ 
sized  defense  firms  have  to  participate  in  defense  R&D.  In  addition,  the  government  must 
establish  technology-innovation  support  programs  for  the  small  and  medium-sized  defense 
firms. 

6.  Reinforcement  of  Arms  Export  Sales 

The  establishment  of  an  improved  export  marketing  strategy  by  the  NDI  is  restricted 
because  customers  are  foreign  governments.  Thus,  the  government-wide  networks  to 
support  defense  exports  must  be  strengthened.  Potential  and  promising  items  have  to  be 
selected  and  developed  to  ensure  competitive  advantages  over  other  countries,  and  the 
basis  of  cooperation  among  governments  has  to  be  strengthened  by  additional  agreements. 
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Korea’s  defense  imports  ranked  the  7th  to  the  9th  largest  in  the  world,  while  Korea’s 
export  market  share  was  just  0.2%  (the  20th  largest)  in  recent  years.  In  the  period  of  2001 
through  2005,  Korea’s  arms  exports  ranked  the  17th  largest  ($337  million),  while  Korea’s 
arms  imports  ranked  the  9th  largest  ($2,561  million).  This  severe  unbalance  of  the  trade  has 
led  the  Korean  NDI  to  shrink.  Operational  support  systems  have  to  be  established  with 
trustworthy,  importing  countries,  and  measures  have  to  be  prepared  to  enhance  price 
competitiveness — such  as  financial  and  tax  support.  Active  cooperation  with  leading 
countries  also  has  to  be  pursued  aggressively  in  order  to  acquire  their  advanced  systems 
and  technologies. 

Conclusions 

This  study  was  conducted  to  investigate  the  international  competitiveness  of  the 
Korean  National  Defense  Industry.  The  characteristics  and  problems  of  various  existing 
competitive  models  for  other  civil  industries  were  reviewed  and  analyzed  to  develop  a  new 
optimized  analysis  model. 

The  new  analysis  model,  the  Pentagon-Defense  8  Factors  model,  fully  takes  into 
account  the  defense  characteristics  that  have  been  derived  to  suggest  the  innovative 
strategy  enhancing  the  international  competitiveness  of  the  Korean  NDI.  As  for  the  Korean 
NDI,  this  model  is  composed  of  5  determinants  (factor  conditions,  demand  conditions,  firms 
strategy  and  rivalry,  related  and  supporting  industries,  and  government  defense  policy)  and 
3  influencers  (defense  industrial  cluster,  defense  policy  of  foreign  nation  and  mutual  relation, 
and  chance). 

The  researcher  studied  the  international  competitiveness  of  the  Korean  NDI  by 
applying  the  P-D8F  model.  Fle  then  suggested  a  possible  reinforcement  strategy  and 
defense  acquisition  policy  to  enhance  the  global  competitiveness  for  the  Korean  NDI  and  to 
effectively  achieve  the  desired  technological  advancement. 

From  this  study,  the  researcher  concluded  that  the  proposed  analysis  model  is  a 
useful  and  practical  one  for  analyzing  and  enhancing  the  international  competitiveness  of 
the  Korean  NDI. 
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to  contracting  and  program  management  issues.  As  a  professor  at  DAU,  he  presents  views 
to  foster  a  more  viable  and  effective  defense  acquisition  management  system.  He  serves 
on  various  DAU  advisory  councils  and  its  academic  review  committee.  Additionally,  he 
provides  independent  consulting  services  to  a  variety  of  industry  clients  relative  to 
procurement,  contract  management,  logistics  and  supply  chain  management. 

As  a  Vice  President  at  CACI  International,  Vincent  was  responsible  for  working  with  senior 
Department  of  Defense  and  Industry  leaders  to  build  long-term  CACI  relationships  and  to 
help  identify  solutions  to  acquisition,  logistics,  and  financial  management  challenges.  His 
strategic  focus  was  an  initiative  to  create  an  integrated  digital  environment  that  will  extend 
the  DoD’s  automated  procurement  system  into  industry  and  into  the  DoD  program 
management  offices,  in  addition  to  implementation  and  training  strategies  for  new  products 
and  services. 

As  a  Vice  President  at  American  Management  Systems,  he  led  a  130-member  business 
unit  responsible  for  the  deployment  and  launch  of  government  and  industry  procurement  and 
contract  management  software  solutions.  His  acquisition  business  solutions  profit  center  was 
responsible  for  implementing  the  DoD’s  Standard  Procurement  System  currently  being 
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management  system  for  industry,  which  was  purchased  by  The  Boeing  Company. 
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technology-based  distance  learning. 

Prior  to  leading  DSMC,  Vincent  had  served  as  the  Logistics,  Ordnance  and  Fleet  Supply 
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(AS  37).  Some  of  his  other  shore-based  assignments  included:  Assistant  Commander  for 
Contracts  at  the  Naval  Air  Systems  Command;  Commander,  Defense  Contract 
Management  Command  International;  Commander,  Defense  Contract  Administration 
Services  Region,  Los  Angeles;  Director,  Contracts  Director  at  Navy  Inventory  Control  Point, 
Mechanicsburg;  Contracting  Officer,  SUPSHIP  Bath,  Maine;  and  Director,  Contracts  Navy 
Supply  Center,  Puget  Sound. 

Vincent  holds  a  Master’s  in  Business  Administration  from  George  Washington  University. 
He  also  is  a  Certified  Navy  Material  and  Acquisition  Professional,  and  is  DAWIA  Level  III 
certified  in  both  Contracting  and  Logistics. 

He  is  President-elect  of  the  National  Contract  Management  Association  and  serves  on  its 
Board  of  Directors  and  Board  of  Advisors.  He  also  serves  on  the  Board  of  Directors,  Navy 
League  National  Capital  Council;  Board  of  Directors,  NDIA  Washington,  DC,  Chapter; 

Board  of  Visitors,  Defense  Acquisition  University;  Board  of  Directors,  Procurement  Round 
Table;  and  as  a  member  of  AFCEA  and  AUSA. 
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Review  Quarterly  now  known  as  the  Defense  Acquisition  Review  Journal.  He  served  as  the  NPS 
member  of  the  Defense  Acquisition  Research  Element  (DARE)  from  1983-1990. 
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Commander  (Ret)  Yoder  entered  the  United  States  Naval  Service  in  1984.  Since  his  commission,  he 
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Abstract 

The  Deputy  Assistant  Secretary  of  the  Navy  for  Research,  Development  & 
Acquisition  (DASN  (RDA)  (Acquisition  Management)),  asked  the  Naval  Postgraduate  School 
(NPS)  to  analyze  the  contracting  out  of  procurement  functions  currently  being  accomplished 
by  Navy,  Marine  Corps,  and  other  Department  of  Defense  (DOD)  Activities.  The  request 
specifically  focused  on  assessing  the  degree  of  effectiveness  and  shortcomings  of  such 
contracting  out  efforts.  This  research  sought  to  answer  these  questions:  Which  contracting 
functions  are  now  being  contracted  out  by  Navy  and  other  DOD  organizations?  How 
effectively  have  contractors  performed  on  these  contracts?  What  metrics  are  being  used 
and  could  be  used  to  assess  the  quality  of  contractor  performance?  Although  the  primary 
focus  of  this  study  is  the  effectiveness  of  contracts  used  to  procure  contracting  services, 
several  interrelated  subjects  have  been  explored.  Aspects  of  inherently  governmental 
functions,  personal  service  relationships,  conflicts  of  interest,  and  legal/ethical  issues  were 
included.  Further,  questions  regarding  the  impact  on  the  contracting  system,  the 
development  of  Contracting  Officers,  the  participation  of  competing  companies  in  the 
marketplace,  training  and  experience  qualifications,  and  agency  procurement  decision¬ 
making  and  policies  were  also  examined. 

Introduction 

Several  factors  have  led  to  an  increased  reliance  upon  the  private  sector  to  provide 
services.  One  of  the  most  critical  factors  has  been  the  lack  of  adequate  numbers  of  civil 
servants  to  perform  the  functions  required  of  buying  organizations.  The  number  of  DOD 
employees  has  been  significantly  reduced  due  to  retirements  or  transfers  to  other  agencies 
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and,  in  certain  metropolitan  areas,  the  number  of  qualified  applicants  available  to  fill  vacant 
positions  has  fallen  to  a  seriously  low  level.  Another  of  the  principal  factors  has  been  to 
reduce  the  cost  of  providing  services.  With  competition  and  a  more  efficient  process  of 
producing  services,  it  is  widely  believed  that  significant  savings  have  accrued.  Another 
factor  has  been  the  ability  to  obtain  certain  skills  which  the  Government  does  not  possess. 
This  has  become  more  critical  as  agencies  have  reduced  the  size  of  their  workforce.  Yet 
another  factor  is  to  obtain  services  on  an  emergency  or  surge  basis.  DOD  has  come  to  rely 
more  and  more  extensively  on  service  contractors  during  military  conflicts.  For  these 
reasons,  some  organizations  have  begun  to  contract  out  selected  contracting  functions 
associated  with  the  acquisition  process.  Further,  some  have  actively  promoted  contracting 
out  efforts  and  see  this  as  an  integral  part  of  their  corporate  strategy.  But,  for  various 
reasons  explored  herein,  some  organizations  have  taken  no  action  to  contract  out 
procurement  functions. 

Methodology 

Sources  involved  in  acquisition  research  were  consulted,  including  the  following: 
reports  issued  by  the  Government  Accountability  Office  (GAO);  theses  and  master’s  degree 
projects  from  students  at  the  Naval  Postgraduate  School  (NPS)  and  the  Air  Force  Institute  of 
Technology  (AFIT);  student  reports  from  the  Naval  War  College,  Army  War  College,  and  Air 
War  College;  reports  and  studies  from  the  RAND  Corp.,  the  Project  on  Government 
Oversight  (POGO),  the  Defense  Science  Board  (DSB),  the  Contract  Management  Institute 
(CMI),  the  Professional  Services  Council  (PSC)  and  the  Logistics  Management  Institute 
(LMI);  papers  presented  at  the  NPS  Annual  Acquisition  Research  Symposium;  and  student 
and  faculty  reports  from  the  Defense  Acquisition  University  (DAU)  and  the  University  of 
Maryland.  Various  periodicals  were  examined,  including  the  Defense  Acquisition  Review 
Journal,  Contract  Management i  Defense  AT&L,  and  the  Journal  of  the  National  Contract 
Management  Association. 

Two  survey  questionnaires  were  used.  The  first  focused  on  participants  at  the  policy 
and  senior  management  levels  and  asked  questions  about  the  broader  issues  involved  in 
contracted  procurement  services.  The  second  survey  focused  on  management  and 
operating  level  personnel  and,  although  some  of  the  same  questions  on  the  first  survey 
were  also  posed,  it  mainly  asked  questions  regarding  the  effectiveness  of  contracts  that  are 
being  or  had  been  used  to  procure  contracting  support  services.  Surveys  were  completed 
by  a  total  of  one  hundred  contracting  professionals  and  thirty-two  program  management  and 
technical  personnel.  A  comparison  of  organizational  affiliation  and  category  of  survey 
participants  is  presented  in  Table  1.  All  thirty-two  program  management  and  technical 
personnel  are  from  the  Air  Force  located  at  Tinker  Air  Force  Base,  Oklahoma.  In  some 
instances,  the  senior  managers  of  a  major  acquisition  organization  provided  collective 
views.  Thus,  the  number  of  individuals  participating  in  the  survey  are  greater  than  the 
number  of  questionnaires  received. 
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Table  1.  Survey  Participants 


Organization 

Policy  and 
Senior 
Management 

Management  and 
Operating  Level 
Personnel 

Total  Survey 
Participants 

Army 

5 

11 

16 

Navy/Marine  Corps 

18 

17 

35 

Air  Force 

5 

2 

7 

Defense  Agencies 

13 

25 

38 

Non-Federal  Agencies 

4 

0 

4 

Air  Force  Prgm  Mgmt  & 
Tech  Personnel 

0 

32 

32 

Totals 

45 

87 

132 

Personal  (face-to-face)  interviews  were  conducted  with  nineteen  senior  contracting 
personnel  (over  80%  of  whom  are  from  the  Navy  Department)  many  of  whom  also 
completed  a  survey.  Phone  interviews  were  conducted  with  fifteen  individuals  and  generally 
included  those  who  had  indicated  willingness  on  their  survey  responses  to  clarify  or  expand 
upon  their  answers.  Throughout  the  study,  the  terms  “procurement”  and  “contracting”  are 
used  interchangeably,  as  are  the  terms  “function,”  “task”  and  “duty.”  The  terms  “contracted 
services,”  “contracted  support  services”  and  “procurement  services”  refer  to  those 
contracting  functions  or  tasks  that  are  typically  performed  by  civil  servants  and  are  now,  or 
could  be,  performed  by  contractor  employees.  “Outsourcing”  refers  to  accomplishment  by 
contractors  and  does  not  include  other  Federal  Agencies.  This  research  work  was 
undertaken  with  the  intent  of  exploring  and  evaluating  only  those  actions  and  efforts  taken 
by  the  Government  in  the  buyer-seller  relationship.  Although  there  are  a  significant  number 
of  companies  that  provide  contracted  support  services  as  well  as  industry  and  professional 
associations  that  have  intimate  knowledge  and  understanding  of  the  process,  the  present 
research  was  limited  to  the  issues  and  problems  experienced  only  by  Government 
personnel. 

A  few  situations  arose  which  impeded  the  numbers  of  surveys  and  interviews  that 
might  otherwise  have  been  obtained.  One  situation  was  the  impression  that  this  study  was 
looking  for  those  contracting  functions  which  could  be  prime  targets  for  contractor 
performance,  which  in  turn  could  lead  to  a  reduction  in  contracting  workforce  personnel.  A 
second  situation  occurred  in  which  agencies  are  contracting  out  some  functions  that  other 
agencies  consider  to  be  inherently  governmental,  and  those  agencies’  officials  felt  that  they 
would  be  criticized  for  having  placed  these  functions  on  contract.  A  third  situation  involved  a 
general  feeling  that  top  agency  management  was  against  placing  contracting  functions  on 
contract,  which  made  our  study  a  moot  point.  Lastly,  some  organizations  simply  said  they 
were  too  busy  to  participate  in  the  research. 

Discussion 

Inherently  Governmental  Functions 

Survey  questions  concerning  inherently  governmental  functions  (IGF)  focused  on 
whether  respondents  were  aware  of  any  functions  considered  to  be  inherently  governmental 
or  exempt  from  competition  that  are,  in  fact,  being  contracted  out  and  the  extent  to  which 
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capability  deficiencies  forced  organizations  to  identify  their  interpretation  of  inherently 
governmental.  Also,  from  a  very  limited  list  of  functions,  survey  participants  were  asked  to 
distinguish  those  they  felt  were  inherently  governmental  as  opposed  to  those  that  were  not. 
Forty  percent  of  the  respondents  stated  that  a  capability  deficiency  had  caused  their 
organization  to  assess  whether  or  not  a  contracting  function  was  inherently  governmental. 
The  shortage  of  Full-Time  Equivalents  (FTEs)  due  to  various  forms  of  attrition  has  been 
exacerbated  in  recent  years.  Retirements  and  personnel  transfers  with  the  resultant  loss  of 
corporate  knowledge  and  expertise  have  forced  organizations  to  rethink  their  position 
regarding  tasks  contractors  can  perform.  The  60%  that  said  they  had  not  made  this 
assessment  are  from  organizations  that  long  ago  decided  that  certain  contracting  tasks  were 
non-IGF  and  were  placed  on  contract,  had  decided  the  entire  function  is  off  limits  to 
contracts,  or  had  sufficient  resources  to  meet  workload  demands.  When  asked  if  they  knew 
about  inherently  governmental  functions  being  contracted  out,  slightly  fewer  than  20% 
acknowledged  that  this  is  happening.  Although  this  may  seem  like  a  small  number,  it  points 
out  that  there  are  functions  being  acquired  on  contract  that  some  view  as  violating  the  rules. 
A  large  part  of  this  could  be  due  to  the  disparity  between  those  who  believe  that  some 
contracting  tasks  are  IGFs  and  those  who  do  not. 

Policy  survey  respondents  were  presented  with  a  limited  list  of  contracting  functions 
and  asked  to  classify  them  as  either  IGF  or  non-IGF.  From  the  replies,  three  categories 
were  established:  (1)  “unanimously”  or  predominantly  IGF,  (2)  predominantly  non-IGF,  and 
(3)  “middle  ground.”  Table  2  displays  the  three  categories. 

Table  2.  Senior  Contracting  Personnel  Views  of 
Inherently  Governmental  vs  Non-lnherently  Governmental  Functions 


Predominantly  Inherently 
Governmental 

“Middle  Ground” 

Predominantly  Non- 
lnherently  Governmental 

Requirements  determination 

Developing  Statements  of  Work 

Structuring  market  research 

Conducting  market  research 

Performing  acquisition  planning 

Developing  solicitation  documents 

Issuing  solicitation  documents 

Developing  and  applying  evaluation  criteria 

Member  of  Source  Selection  Evaluation 

Board 

Evaluation  of  proposals/offers 

Performing  cost  and  price  analysis 

Negotiating  contract  prices,  terms 
&  conditions 

Structuring  &  approving  incentive 
plans 

Preparing  price  negotiation  memoranda 

Awarding  contracts 

Negotiating  contract  modifications 

Determining  cost  allowability 

Exercising  options 

Assessing  contractor  performance 

Implementing  action  based  on 

ACQUISITION  RESEARCH:  CREATING  SYNERGY  FOR  INFORMED  CHANGE 


-411  - 


contractor  performance 

Accepting  or  rejecting  goods  & 
services 

Terminating  contracts 

Preparing  contracts  for  closeout 

If  no  more  than  75%  or  no  fewer  than  25%  selected  a  function  as  either  IGF  or  non- 
IGF,  the  function  was  placed  in  the  “middle  ground”  and  open  to  interpretation.  All  of  the 
IGFs  involve  some  type  of  Contracting  Officer  determination  and/or  decision  which  are 
considered  by  almost  everyone  to  be  within  the  Government’s  purview.  The  two  functions 
identified  as  essentially  non-IGF  have  been  performed  by  contractors  for  several  years  and 
now  seem  to  be  the  accepted  norm.  The  “middle  ground”  functions  are  basically  the  area  of 
debate.  Functions  listed  close  to  the  left  of  the  box  received  the  majority  of  IGF  responses, 
while  those  functions  close  to  the  right  received  the  majority  of  non-IGF  responses. 
Structuring  market  research  is  closely  related  to  conducting  market  research  and  developing 
solicitation  documents  is  often  distinguished  as  a  support  effort.  These  two  functions  could 
have  easily  slipped  into  the  non-IGF  category.  Performing  cost  and  price  analysis  is  the 
function  that  seems  to  generate  the  most  heated  debate.  Many  believe  it  to  be  integrally 
involved  in  either  the  negotiation  process  or  a  Contracting  Officer’s  determination  of  price 
reasonableness,  or  both,  and  should  not  be  contracted  out.  Further,  considered  by  several 
to  be  a  core  capability,  it  is  also  judged  to  be  one  of  the  weakest  skills  held  by  the 
Government  and  badly  in  need  of  significant  training  and  hands-on  experience.  Although 
tending  toward  the  IGF  side,  acquisition  planning,  evaluation  of  offers  and  preparing  price 
negotiation  memoranda  are  just  as  easily  viewed  by  many  as  non-IGF. 

Personal  Services  Relationships 

Over  85%  of  the  respondents  felt  that  there  are  potential  problems  with  personal 
services  relationships  when  contracting  for  procurement  services.  On  the  one  hand,  there  is 
an  overwhelming  feeling  that  contractor  employees  need  to  be  co-located  with  Government 
contracting  personnel  in  order  to  effectively  accomplish  their  work.  On  the  other,  this  close 
proximity  presents  the  very  ingredients  that  lead  to  personal  services  situations.  Many  of 
the  respondents  felt  that  because  Government  and  contractor  specialists  had  to  work 
together  so  intimately,  such  a  relationship  was  inevitable.  Not  directly  causing  a  personal 
services  dilemma  but  linked  to  the  close  working  relationship  are  the  circumstances 
involving  “acceptance”  of  the  service  or  work  product  performed  by  contractors.  Products  or 
goods  have  a  very  definitive  acceptance  or  rejection  process  that  requires  specific 
Government  action.  Services  do  not  have  similar  acceptance  procedures.  In  the  case  of 
contracting  functions,  who  accepts  the  contractor’s  work  product  and  how  formal  is  the 
process?  If  the  Government  proceeds  to  use  work  packages  submitted  by  contractors,  de 
facto  acceptance  has  probably  occurred  without  an  overt  action.  At  this  point,  the 
Government  could  certainly  be  liable  for  the  decisions  made  based  on  the  contractor  input 
that  might  later  prove  to  be  faulty  and  damaging.  None  of  the  respondents  voiced  an 
opinion  that  the  prohibition  on  personal  services  contracts  should  be  eliminated;  however,  if 
asked,  it  is  suspected  that  a  majority  would  willingly  agree  with  the  removal  of  certain 
aspects  of  the  personal  services  restriction. 

Conflicts  of  Interest 

Survey  participants  responded  with  numerous  conflict  of  interest  situations  they 
believed  could  easily  occur  unless  precautionary  measures  were  instituted.  Access  to 
company  proprietary  and  business  sensitive  information,  competing  in  cases  where  firms 
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participated  in  developing  requirements,  an  actual  or  perceived  ability  to  influence 
procurement  actions,  biases  against  certain  companies  for  obvious  or  even  unknown 
reasons,  insight  into  the  Government’s  requirements  process,  mergers  and  acquisitions  that 
cause  questionable  affiliations,  and  other  similar  opportunities  to  inappropriately  affect 
Government  procurement  were  cited  as  potential  problems  with  contractors.  Some 
respondents  remarked  that  conflicts  of  interest  could  also  easily  occur  with  civil  servants 
working  so  closely  with  contractor  employees.  Differences  in  compensation,  future 
employment  opportunities,  and  personal  friendships  with  contractor  employees  that  may 
even  have  originated  when  both  worked  for  the  Government  were  observed  as  some  of  the 
conflict  of  interest  situations  in  which  civil  servants  could  easily  become  embroiled.  A  recent 
Government  Accountability  Office  (GAO,  2008)  report  concluded  that  when  contractors  are 
performing  duties  closely  supporting  inherently  governmental  functions,  risks  exist  that  could 
result  in  loss  of  Government  control  and  decision-making.  The  Government  and  contractors 
are  both  responsible  for  ensuring  that  measures  are  in  place  to  prevent  conflicts  of  interest. 

Legal  and  Ethical  Issues 

This  research  asked  if  there  are  any  legal  issues  or  impediments  to  which  over  85% 
responded  in  the  affirmative.  Respondents  are  concerned  about  three  primary  areas:  (1) 
violating  the  prohibition  against  placing  inherently  governmental  functions  on  contract,  (2) 
averting  personal  services  relationships,  and  (3)  avoiding  conflicts  of  interest.  They  state 
that  proper  non-disclosure  agreements,  financial  disclosure  statements,  and  vigorous 
application  of  safeguards  and  security  mechanisms  are  critical  to  prevention.  It  is  well 
known  that  civil  and  criminal  penalties  await  Federal  employees  if  they  violate  statutes 
prohibiting  unethical  and  improper  behavior  in  the  execution  of  their  responsibilities.  This  is 
generally  not  true  if  contractor  employees  performing  procurement  functions  for  the 
Government  act  in  this  manner.  Several  situations  concerning  contractor  personnel 
involved  with  financial  conflicts  of  interest,  impaired  impartiality,  misuse  of  information, 
misuse  of  authority  and  misuse  of  Government  property  have  transpired.  There  has  been 
some  discussion  about  the  suitability  of  changing  the  laws  to  make  them  applicable  to 
contracted  employees  when  they  are  working  on  behalf  of  the  Government.  Some  suggest 
that  instead,  contract  clauses  should  incorporate  such  requirements.  At  the  very  least, 
contractor  personnel  should  be  held  liable  and  accountable  for  actions  taken  in  their 
capacity  as  “Government  agents.”  Whether  this  is  accomplished  statutorily  or  through 
contractual  clauses,  some  action  in  this  direction  is  absolutely  crucial. 

Ethics  was  brought  up  on  the  Policy  and  Senior  Management  questionnaire  and  a 
significant  majority  of  the  respondents  believe  that  ethical  issues  are  associated  with 
contracting  for  procurement  services.  To  some,  this  took  the  form  of  organizational  conflicts 
of  interest  while  to  others  it  involved  contractor  access  to  procurement  sensitive  information 
and  the  opportunity  to  take  unfair  advantage  of  the  system.  Most  of  the  respondents 
expressed  concern  about  contractor  loyalties  and  motivations  which  might  impair  their 
objectivity  and  impartiality  when  acting  as  an  “agent”  for  the  Government.  The  judgment 
and  interpretation  of  the  laws  and  regulations  by  Government  employees  should  not  be 
replaced  by  contractor  personnel.  The  best  interests  of  the  Government  should  be 
foremost  in  any  action  taken  by  someone  in  the  contracting  process.  Misgivings  can  easily 
arise  when  contractor  employees  take  these  actions.  Trust  in  the  system  can  be  easily  and 
seriously  jeopardized. 
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Procurement  Functions  Contracted  Out 


Over  recent  years,  the  numbers  and  types  of  functions  contracted  out  has  greatly 
increased.  Some  organizations  have  become  concerned  that  too  great  a  percentage  of 
workforce  positions  are  filled  with  contractor  employees  and  have  begun  to  develop  plans  to 
reduce  that  percentage.  Close  to  60%  of  survey  participants  indicated  that  at  least  some 
procurement  functions  are  being  contracted  out.  Policy  and  Senior  personnel  have  fewer 
situations  of  contracting  out  while  Management  and  Operating  Level  Personnel  are 
predominantly  from  organizations  that  are  contracting  out  procurement  functions.  When 
isolating  the  Defense  Agencies  from  the  Services  and  others,  however,  that  number 
exceeded  85%.  When  looking  only  at  Navy  and  Marine  Corps  organizations,  that  number 
dropped  slightly  below  35%.  These  findings  are  consistent  with  other  studies,  which  found 
the  Defense  Agencies  with  the  highest  ratio  of  contracted  support  services  and  the  Navy 
and  Marine  Corps  with  the  lowest.  When  asked  which  functions  are  involved,  contract 
closeout  was  identified  with  the  greatest  frequency.  This  was  true  for  all  Services  and  the 
Defense  Agencies.  Policy  and  Senior  Management  personnel  tended  to  cite  market 
research,  acquisition  planning,  drafting  policy,  developing  evaluation  criteria,  evaluation  of 
offers,  and  requirements  development  as  the  predominant  functions  contracted  out.  Some 
policy/senior  personnel  did  state,  however,  that  all  functions  performed  by  1102s,  except  for 
inherently  governmental  functions,  are  placed  on  contract.  The  percentage  of  Management 
and  Operating  Personnel  who  reported  functions  contracted  out  in  their  organizations  was 
much  higher  and  included  all  contract  specialist  functions  in  support  of  the  Contracting 
Officer  in  both  pre-award  and  post-award  phases.  Most  were  careful  to  explain  that  the 
approvals,  determinations  and  decisions  made  by  Contracting  Officers  were  not  included. 

The  most  prevalent  reasons  cited  for  contracting  out  are  the  lack  of  organic 
resources  to  meet  workload  demands  and  the  lack  of  needed  skills  or  expertise  in  certain 
areas.  The  continual  downsizing  and  freezes  on  hiring  new  personnel  over  the  last  several 
years,  together  with  an  increase  in  the  workload,  has  severely  strained  the  contracting 
community.  In  certain  geographical  areas,  there  is  constant  turnover  and  an  inability  to  fill 
1102  vacancies  with  qualified  applicants.  In  some  cases,  contractors  are  used  to  obtain 
exposure  to  business  concepts  and  insight  into  commercial  practices  and  technology. 

Some  stated  that  hiring  contractors  is  generally  easier  and  faster  than  trying  to  obtain 
Federal  employees  through  the  cumbersome  civil  service  personnel  process.  Contractors 
provide  greater  flexibility  in  adjusting  to  workload  fluctuations,  particularly  due  to  surge 
situations.  They  can  also  provide  continuity  to  those  cases  in  which  organizations  are 
experiencing  a  very  high  turnover  of  contract  specialists. 

The  most  common  reason  for  not  contracting  out  procurement  functions  is  that  the 
organization  believes  all  aspects  of  contracting  are  inherently  governmental.  Even  if  the 
organization  did  not  consider  contracting  inherently  governmental,  there  was  a  preference 
against  contracting  out,  especially  if  interchangeability  of  personnel  or  adaptability  was 
threatened.  In  smaller  offices,  Contracting  Officers  are  performing  most  contract  specialist 
functions  that  would  be  difficult  to  untangle  from  their  decision-making  functions.  In  some 
cases,  the  organization  claimed  that  sufficient  resources  existed  to  meet  workload 
requirements  or  that  outsourcing  the  functions  did  not  provide  any  additional  advantages.  A 
few  organizations  indicated  that  the  potential  for  conflicts  of  interest  or  other  ethical 
problems  had  steered  them  away  from  using  contractors.  In  a  couple  of  cases,  respondents 
felt  that  contractors  were  more  expensive  than  Government  employees.  One  scenario  that 
has  been  cause  for  concern  is  contractor  default.  Buying  offices  that  have  turned  over 
contract  specialist  workload  to  contractor  employees  may  be  in  a  difficult  situation  if  the 
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contractor  is  terminated  for  default.  It  would  be  arduous  to  explain  to  the  buying  office’s 
customers  that  procurement  actions  are  delayed  because  the  contractor(s)  defaulted.  If  a 
significant  portion  of  that  office’s  workforce  consisted  of  contractor  employees,  the  added 
burden  may  be  too  overwhelming  for  the  civil  servants  in  the  office. 

Effectiveness  of  Contracted  Services 

Management  and  Operating  Level  Personnel  expressed  overwhelmingly  that 
contracts  for  procuring  contracting  functions  were  highly  effective  or  somewhat  effective.  A 
deeper  analysis  of  the  results,  however,  shows  that  most  of  the  respondents  indicating  a 
positive  effectiveness  of  contracts  based  their  evaluation  on  a  simple  objective  set  of 
metrics.  First,  did  contractor  performance  allow  the  command  to  meet  its  mission,  and 
second,  was  overall  performance  good  enough  to  consider  the  contractor  for  future  work. 
The  heart  of  the  rudimentary  metric  gets  at  the  most  basic  rationale  for  contracting  out  any 
function,  getting  the  job  done  and  doing  it  satisfactorily.  Responses  are  based  primarily  on 
criteria  that  find  their  basis  in  whether  the  contractor  allowed  the  activity  or  business  unit  to 
achieve  its  mission  or  productivity  goals,  and  additionally,  whether  the  contractor  had  any 
significant  performance  problems  that  would  preclude  them  from  being  considered  as  a 
candidate  for  future  award  of  similar  work.  No  other  criteria  for  measuring  effectiveness 
were  being  systemically  applied.  What  is  challenging  for  contract  managers  is  that  metrics 
are  difficult  to  capture  in  any  detailed  objective  format.  The  subjective  nature  of  this  type  of 
assessment  by  the  population  surveyed  is  very  common.  It  is,  however,  a  measure  that  can 
be  captured  by  Past  Performance  Information  (PPI)  systems,  and  the  Contractor 
Performance  Assessment  Reporting  System  (CPARS)  currently  in  use  within  DOD. 

Limitations  on  Effort  Contracted  Out 

Three  aspects  of  limitations  on  contractors  performing  procurement  functions  were 
explored:  (1)  the  duration  of  contracted  support  effort,  (2)  whether  contractors  should  be 
physically  located  with  Government  personnel  performing  the  same  tasks,  and  (3) 
percentage  limitations  on  the  amount  of  procurement  effort  that  could  be  contracted  out. 

The  first  area  was  presented  on  the  policy  survey  while  the  latter  two  areas  were  presented 
on  the  management/operating  level  personnel  surveys. 

Duration 

Over  60%  of  the  respondents  believed  these  contracts  should  be  of  a  temporary 
nature.  Organizations  which  tended  to  currently  have  more  procurement  functions 
contracted  out  were  split  over  the  idea  of  temporary  versus  permanent.  The  proponents  for 
temporary  contracts  felt  it  should  only  be  used  for  surge  or  emergent  requirements  while  the 
Government  recruits  and  trains  organic  resources.  Those  advocating  a  more  permanent 
duration  felt  it  will  be  several  years,  if  ever,  before  Government  resource  requirements  are 
met,  thus  a  lasting  contractual  relationship  should  be  established.  Some  feel  that  certain 
tasks,  such  as  contract  closeout,  have  such  a  low  priority  they  might  never  be  appropriately 
completed  without  contractor  support  and  should  be  made  permanent  regardless  of  in- 
house  resource  levels.  It  appears  to  the  researchers  that  almost  all  the  reasons  cited  in  the 
Federal  Acquisition  Regulation  {FAR)  as  valid  alternatives  for  the  use  of  advisory  and 
assistance  services  (A&AS)  contracts  seem  to  imply  temporary  situations.  Obtaining  advice, 
points  of  view,  opinions,  special  knowledge,  alternative  solutions,  support  to  improve 
operations,  and  assistance  with  more  efficient  and  effective  operation  of  managerial  or 
hardware  systems  all  give  the  impression  that  agency  management  would  acquire  these 
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“consultant”  services  on  an  as-needed  basis.  Further,  the  language  does  not  give  the 
impression  that  the  performance  of  routine  tasks  on  a  day-to-day  basis  is  the  intent  of 
employing  this  capability.  If  management  policy  is  to  utilize  contracted  support  only  if  we 
must,  then  certainly  all  contracts  for  this  support  should  be  viewed  as  temporary  until  no 
longer  required.  If  management  policy  is  to  leave  this  up  to  each  individual  organization  to 
decide,  then  the  types  of  functions  they  perform  and  their  overall  view  of  the  procurement 
responsibility  will  drive  their  choice. 


Co-location 


Closely  related  to  the  matter  of  personal  services  relationships  during  contract 
performance  is  the  location  of  contractor  employees  when  they  are  carrying  out  their  duties. 
An  overwhelming  75%  majority  of  respondents  emphatically  expressed  the  need  for  close 
communications  on  a  face-to-face  basis  between  all  members  of  the  acquisition  team.  The 
professional  interaction  that  will  occur  through  physical  proximity  outweighs  any  risks  that 
might  surface.  The  day-to-day  working  conditions  requiring  communication,  efficient 
interaction,  responsive  feedback/input,  and  professional  interface  to  advance  learning  and 
understanding  all  support  a  blended  workforce.  Past  attempts  at  separation  have  shown 
this  to  severely  hinder  smooth  accomplishment  of  the  contracting  functions.  The  25%  who 
said  contractor  employees  should  not  be  in  the  same  spaces  as  civil  servants  were  not 
denying  the  benefits  of  personal  interaction  but  rather  were  implying  that  the  risks  are  more 
than  should  be  accepted.  Access  to  sensitive  and  proprietary  data,  security  considerations, 
and  the  potential  for  conflicts  of  interest  was  of  real  concern.  Personal  service  concerns 
appear  to  be  diminishing. 


Percentage  of  Contracted  Services 


The  last  area  related  to  restrictions  on  the  extent  of  contractor  participation  is  the 
idea  of  confining  the  percentage  of  effort  that  activities  can  place  on  contract.  A  Defense 
Acquisition  University  (DAU)  report  recommended  that  each  contracting  activity  be  limited  to 
no  more  than  25%  of  their  workforce  that  may  be  contracted  out.  The  researchers  decided 
to  ask  not  only  where  the  percentage  limitation  on  the  workforce  should  be  established,  but 
also  what  percentage  of  the  workload  should  be  limited  to  outsourcing.  Regarding 
workforce,  just  over  50%  of  the  respondents  said  that  it  should  be  under  25%  and  the  total 
reaches  three-fourths  of  the  respondents  when  one  goes  to  a  50%  limitation.  Just  over  45% 
of  the  respondents  believe  that  contracting  out  should  be  constrained  to  under  50%  of  the 
total  workload  while  that  percentage  increases  to  65%  if  half  of  the  workload  is  the  limitation. 

Experience  and  Training  Requirements 

The  DOD  acquisition  workforce  has  been  criticized  over  the  years  for  its  lack  of  skills, 
knowledge  and  abilities  to  execute  its  responsibilities.  The  Report  of  the  Commission  on 
Government  Procurement,  the  Packard  Commission  report  and  several  other  studies  from 
similar  groups  have  pointed  to  the  need  for  a  professional  workforce  meeting  minimum 
standards  established  for  education,  training  and  experience.  Recommendations  from 
these  studies  served  as  the  impetus  that  created  the  Defense  Acquisition  Workforce 
Improvement  Act  (DAWIA)  and  led  to  the  standards  in  place  today.  It  would  be  justifiable  to 
expect  that  anyone  performing  contracting  functions  for  DOD  should  meet  these  standards 
and  qualifications.  Should  this  expectation  be  extended  to  contractor  personnel  performing 
procurement  functions  for  the  Government?  And  if  so,  how  difficult  would  it  be  to  impose 
and  enforce  DAWIA  standards  on  contractors?  This  was  the  issue  explored  in  both  surveys. 
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The  Policy  and  Senior  Management  survey  asked  about  DAWIA  implications  and  whether 
DAWIA  should  be  imposed  on  contractors,  while  the  Management  and  Operating  Personnel 
survey  queried  participants  as  to  the  level  of  difficulty  encountered  if  an  attempt  were  made 
to  compel  contractors  to  comply  with  DAWIA.  Seventy  percent  of  the  policy  survey 
respondents  felt  there  are  DAWIA  implications  whereas  about  65%  felt  that  DAWIA  or 
DAWIA-like  requirements  should  be  imposed.  Forty-five  percent  of  the 
management/operating  personnel  said  it  would  be  difficult  or  very  difficult  to  impose  and 
enforce  DAWIA  requirements,  while,  in  contrast,  over  45%  noted  that  it  would  be  easy. 
Although  not  an  overwhelming  majority,  most  participants  felt  that  DAWIA  is  important  to  the 
issue  of  using  contracted  employees. 

The  argument  for  application  of  DAWIA  includes  the  beliefs  that  these  requirements 
are  critical  elements  in  the  performance  of  complex  functions  to  ensure  individuals  have  the 
ability  to  think  logically,  act  competently,  stay  current  in  the  field,  meet  contractual 
expectations,  and  perform  in  a  proficient  manner.  Proponents  believed  that  contractors 
should  be  held  to  the  same  standard  of  competence  as  DOD  personnel.  Those  opposing 
application  of  DAWIA  claim:  it  will  cost  the  Government  an  additional  expense  for 
contractors  to  meet  the  standards;  most  contractor  employees  are  former  Government 
personnel  and  probably  already  have  these  certifications;  DAWIA  is  a  statutory  requirement 
placed  on  the  Government  and  not  industry;  some  are  already  using  DAWIA  standards  as 
an  evaluation  criterion  in  source  selections  or  use  DAWIA  language  in  statements  of  work  to 
describe  desired  labor  categories,  and  it  will  inhibit  competition.  There  appear  to  be  valid 
arguments  on  both  sides.  It  seems  as  if  the  argument  for  DAWIA  focuses  primarily  on  the 
level  of  skills  and  competencies  personnel  should  hold,  while  the  argument  against  mainly 
suggests  that  DAWIA  qualifications  are  already  being  used  to  a  certain  extent  and  to  push 
any  further  would  be  costly  to  the  Government.  One  might  conclude  that  if  DAWIA  is 
already  being  used  in  this  environment  to  one  extent  or  another,  full  application  of  this 
qualification  should  not  be  a  difficult  stretch.  Further,  although  there  may  be  added  expense 
to  the  Government  to  bring  contractor  employees  up  to  a  certain  level  of  competence,  failure 
to  do  so  may  be  even  more  costly  in  the  less-than-satisfactory  performance  of  contracting 
functions,  even  though  there  might  be  Government  oversight. 

Impact  on  the  Contracting  System 

The  research  examined  the  affect  contracted  procurement  services  might  have  on 
the  contracting  system  by  looking  at  three  aspects:  (1)  the  development  of  future 
Contracting  Officers,  (2)  the  development  of  procurement  options  by  agency  management, 
and  (3)  the  extent  to  which  companies  might  not  want  to  participate  in  Government 
procurement.  The  first  two  areas  were  addressed  on  both  surveys  while  the  last  was  asked 
only  on  the  policy  survey. 


Contracting  Officers 


Over  65%  of  the  respondents  felt  that  contracting  out  procurement  functions  could 
have  a  damaging  impact  on  the  development  of  future  Contracting  Officers,  slightly  over 
20%  felt  it  would  not  have  any  affect,  and  less  than  10%  thought  it  might  have  a  beneficial 
impact.  Respondents  believing  a  damaging  impact  could  occur  pointed  to  the  critical  need 
to  nurture  and  cultivate  a  competent  and  professional  workforce.  Significant  contract 
specialist  experience  is  needed  to  progress  through  the  basic  and  intermediate  levels  to 
ultimately  reach  the  advanced  and  expert  proficiency  levels  required  of  Contracting  Officers. 
One  needs  to  be  exposed  to  the  broadest  cross-section  of  contracting  tasks  that  permits 
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development  of  the  critical  thinking  skills  and  competencies  so  fundamental  to  making 
complex  Contracting  Officer  decisions.  It  was  argued  that  contract  specialists  need  to 
experience  the  various  avenues  one  might  pursue  in  accomplishing  a  particular  objective. 
They  need  to  grasp  the  underlying  mechanics  and  inner  workings  and,  in  fact,  they  need  to 
fail  from  time-to-time  to  discover  the  weaknesses  and  risks  surrounding  particular  courses  of 
action.  They  need  to  rotate  through  various  assignments  in  contracting  to  undergo  and  be 
exposed  to  the  specific  facets  these  duties  offer.  Contract  specialists  need  to  be  mentored. 
Not  only  must  they  eventually  acquire  strong  managerial  skills  but  they  must  also  develop 
leadership  capabilities.  The  Government  acts  through  its  Contracting  Officers  and,  by 
extension,  its  contract  specialists.  They  are  the  “face  to  industry”  with  which  the 
Government  speaks  and  acts.  They  exhibit  authority,  execute  responsibilities,  create 
relationships  and  perform  duties  all  as  part  of  the  Government’s  side  of  the  buyer-seller 
relationship.  All  of  these  abilities  are  accumulated  on  the  job,  integrated  with  appropriate 
levels  of  training  and  education.  Many  would  argue  that  the  “culturing”  acquired  through 
mentoring  must  be  achieved  by  interaction  with  a  Government  workforce,  while  others  would 
assert  that  capable  and  adept  contractor  personnel  can  greatly  assist  to  the  same  degree.  It 
can  be  rationalized  that  the  existence  of  contractor  personnel  working  as  contract  specialists 
in  a  Government  organization  can  bring  new  dimensions  to  the  performance  of  procurement 
duties.  An  important  ingredient  frequently  missing  in  buying  offices  is  the  sensitivity  and 
understanding  of  commercial  and  industrial  procurement  practices  that  work  well,  or  do  not 
function  well,  in  the  business  world.  Sound  business  methods  and  the  decisions  that  result 
from  genuine  business  thinking  are  vital  to  any  “business”  even  if  it  is  the  business  of 
Government  procurement.  The  research  has  suggested,  however,  that  most  of  the 
contractor  staff  employed  to  perform  contracting  functions  are  former  Government 
acquisition  personnel  who  have  had  little,  if  any,  industry  experience  to  bring  to  the 
procurement  table.  Some  have  even  complained  that  the  Government  contracting 
knowledge  they  do  bring  is  outdated  and  lacks  currency.  The  business  manager’s  role 
required  of  contracting  professionals  today  differs  from  the  contracting  technician’s  role  of 
even  a  few  years  ago.  The  research  thus  far  has  suggested  that  contractor  employees 
performing  procurement  functions  are  dedicated,  trustworthy  and  reliable.  Their  loyalty  to 
the  US  is  unquestioned.  However,  because  their  livelihood  is  derived  from  a  private  entity, 
there  could  be  an  inkling  of  suspicion  on  the  part  of  Government  managers  that  these 
employees  might  not  always  be  placing  the  best  interests  of  the  Government  ahead  of  all 
others.  This  thought  could  carry  into  the  interactions  between  Government  contract 
specialists  and  contractor  personnel  where  proposed  courses  of  action  might  not  be  the 
most  beneficial  to  the  Government.  Ruling  out  ignorance  or  incompetence  for  the  moment, 
some  would  question  that  contractor  employees  would  investigate  all  viable  alternatives 
before  coming  forth  with  a  proposed  solution,  particularly  if  one  or  more  alternatives  were 
perceived  as  detrimental  to  their  company. 

Procurement  Options 

One  of  the  major  concerns  regarding  procurement  of  contracting  functions  is  the 
long-term  affect  this  would  have  on  the  ability  of  agency  management  to  develop  and 
consider  procurement  options.  Over  40%  of  the  respondents  believed  that  this  will  expand 
the  procurement  options  while  less  than  25%  believed  it  will  limit  such  options.  Almost  30% 
felt  it  would  neither  limit  nor  expand  the  procurement  options.  Most  notably,  Policy  and 
Senior  Management  personnel  mostly  believed  that  it  will  enhance  options.  Arguments  can 
be  made  for  both  the  notion  that  options  are  expanded  and  the  concern  that  options  are 
limited.  Several  valid  points  support  the  former.  Contractors  can  free  up  Government 
personnel  to  perform  more  complex  or  value  added  tasks;  they  can  come  with  specific  skills 
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and  expertise  to  supplement  Government  weaknesses;  they  usually  operate  in  a  competitive 
environment  and,  therefore,  have  developed  innovative  approaches  that  can  be  shared  with 
the  Government;  they  are  not  as  stove-piped  in  their  thinking  and  training,  and  they  come  at 
a  time  when  in-house  resources  and  capabilities  appear  to  be  extremely  low.  One  of  the 
chief  reasons  set  forth  by  proponents  of  the  “expand”  position  is  that  contractor  employees 
bring  knowledge  of  industry  best  practices  and  business  concepts  that  can  be  shared  with 
contract  specialists.  This  could  be  a  way  of  introducing  commercial  procedures  and  actions, 
which  might  greatly  assist  in  executing  a  more  effective  and  efficient  procurement  system. 
Consider,  however,  the  sources  from  which  these  contractor  personnel  might  come.  Many 
buying  organizations  claim  that  their  contractors  use  former  Government  civil  servants  and 
military  personnel,  which  increases  their  ability  to  perform  contracting  functions.  But  this 
almost  assures  that  they  will  have  had  little  industry  experience.  Even  those  individuals  who 
have  previously  worked  for  companies  holding  prime  contracts  with  the  Federal  Government 
have  been  in  the  Contracts  Division  and  interfacing  directly  with  their  Federal  counterparts. 
Not  until  you  get  to  someone  with  experience  in  the  Procurement  or  Subcontracts 
Department  of  a  company  will  you  find  an  individual  likely  to  be  immersed  in  business 
practices.  If  employees  come  from  outside  this  realm,  they  probably  have  had  little 
“Government”  experience  with  which  to  execute  their  contracting  responsibilities  and  will 
most  likely  have  a  significant  learning  curve  in  becoming  familiar  with  the  Federal 
procurement  world.  The  prime  reasons  set  forth  by  those  who  claim  it  will  limit  options 
involve:  (1)  a  belief  that  a  contractor’s  objectivity  will  be  questioned  because  they  might  not 
always  have  the  Government’s  best  interests  at  heart;  (2)  the  restrictive  nature  of  firewalls 
and  other  safeguards  necessary  to  ensure  prevention  of  conflicts  of  interest;  (3)  a  denial  of 
valuable  training  and  experience  for  junior  Government  personnel;  (4)  contractual  limitations 
placed  on  organizations  as  to  how  they  can  use  contractor  employees;  (5)  experience  with 
contractors  where  little  assistance  with  options  was  provided;  and  (6)  fewer  experienced  civil 
servants  available  to  adequately  assess  and  evaluate  contractor  performance.  Those  who 
felt  there  would  be  no  affect  on  the  ability  to  develop  procurement  options  essentially 
believed  that  talented  individuals  will  make  a  considerable  contribution  regardless  of  their 
origin:  Government  or  contractor.  Further,  they  point  out,  it  really  is  up  to  management  to 
decide  how  to  act  on  specialists’  input,  no  matter  what  the  source 

Market  Participation 

The  extent  to  which  companies  are  willing  to  participate  in  procurements  when  one 
or  more  other  companies  are  involved  in  performing  contracting  functions  for  the 
Government  should  be  of  some  concern.  The  health  of  the  industrial  base  is  often 
measured  by  the  amount  and  nature  of  competition.  If  companies  become  suspicious  of 
their  treatment  in  Government  competitions,  their  eagerness  to  continue  may  be  dampened. 
The  result  could  be  that  less  information  is  forthcoming,  particularly  confidential  and  private 
data,  or  worse,  that  companies  withdraw  from  Government  competitions  altogether.  The 
“large”  contractor  dependent  on  Government  contracts,  particularly  if  it  is  a  sole  source,  is 
less  likely  to  disengage  but  could  potentially  restrict  the  flow  of  information.  Companies  with 
a  significant  amount  of  commercial  business  in  addition  to  public  contracts  could  very  well 
decide  to  no  longer  stay  connected  with  Government  procurement.  Over  one-half  of  the 
respondents  are  concerned  about  a  negative  impact.  Most  of  the  respondents  from  the 
Services,  over  75%,  believed  that  a  negative  affect  could  occur,  but  no  one  from  the 
Defense  Agencies  held  this  belief.  Many  of  those  from  the  Defense  Agencies  have  had 
experience  with  contracted  procurement  services  and  can  report  their  views  from  actual 
situations.  The  primary  concern  from  those  expressing  a  negative  impact  centers  on  the 
fairness  and  objective  treatment  of  competing  firms.  The  inappropriate  use  of  proprietary 
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data,  biased  evaluations,  and  undue  influence  by  private  companies  all  can  lead  to  distrust 
in  the  integrity  of  the  contracting  process.  Some  would  point  out  that  all  of  these  abuses 
could  occur  with  Government  personnel  as  well  but  Federal  laws  impose  civil  and  criminal 
sanctions  on  these  individuals  which  is  not  the  case  with  contractor  employees.  This  affect 
is  subtle  and  difficult  to  measure  because  it  entails  industry  perceptions.  It  is  unlikely  that 
most  companies,  if  questioned,  would  cite  their  distrust  of  the  system  as  the  reason  for  non¬ 
participation. 


Integrity  of  the  Contracting  Process 

This  area  was  brought  out  on  both  surveys.  Although  some  of  the  respondents 
asserted  that  the  best  way  to  ensure  integrity  is  to  completely  avoid  using  contractors,  most 
provided  thoughts  and  ideas  they  felt  would  help  to  maintain  a  robust  contracting  system. 
Many  pointed  to  proactive  efforts  that  will  maintain  necessary  components  of  integrity. 
Sensible  policies  that  discuss  the  legal,  ethical,  and  practical  aspects  are  critical.  Integrity 
has  to  do  with  the  image  and  reputation  of  the  system.  The  actions,  or  inactions,  resulting 
from  “challenges”  to  the  system,  such  as  fraudulent  or  abusive  events,  shape  the  character 
and  personality  of  the  system.  How  the  Government  handles  a  breach  of  our  laws  and 
ethics  policies,  both  by  civil  servants  and  contractors,  is  highly  visible  and  reaches  to  the 
heart  of  our  moral  fiber.  It  is  the  responsibility  of  every  member  of  the  acquisition  team  to  do 
his  or  her  utmost  to  preserve  the  highest  quality  of  our  collective  personality  and  culture. 
Certainly,  safeguards  and  precautions  are  vital.  Methods  to  discourage  or  prevent  conflicts 
of  interest,  illegal  actions,  and  other  similar  activities  must  be  in  place.  Internal  Government 
efforts,  such  as  oversight,  audits,  reviews,  surveillance,  awareness  training,  and  firewalls 
can  and  should  be  used.  External  efforts  focused  on  contractors  are  also  important  and 
could  include  non-disclosure  statements  to  protect  sensitive  data,  financial  disclosure,  and 
ethical  and  integrity  certifications.  Going  a  step  further,  it  could  be  argued  that  contractor 
employees  who  have  been  entrusted  with  the  same  responsibilities  as  civil  servants  should 
face  the  same  consequences  for  contraventions  of  that  trust.  Civil  and  criminal  sanctions 
should  apply  equally  to  all  who  are  accountable  for  public  endeavors.  A  very  recent  (March 
2008)  GAO  report  cited  the  need  for  additional  conflict  of  interest  safeguards  for  contractor 
employees  who  work  alongside  DOD  civil  servants. 

Procurement  Policy 

This  topic  was  broached  to  the  Policy  and  Senior  Management  personnel  in  both  the 
surveys  and  interviews.  Over  60%  believe  that  a  policy  statement  would  be  extremely 
helpful  in  clarifying  top  management’s  position  on  all  aspects  of  this  issue.  Their  main 
concern  is  the  range  in  difference  of  opinion  concerning  the  definition  of  inherently 
governmental  functions  and  a  more  direct  application  to  contracting  functions  is  needed. 

The  35%  who  do  not  believe  a  policy  statement  is  necessary  believed  that  sufficient  policy 
and  guidance  already  exists  and  any  further  language  on  the  subject  would  probably  serve 
to  make  things  more  restrictive.  Key  elements  that  should  be  included  if  such  a  policy  were 
published  varied.  Respondents  felt  a  policy  should  be  very  flexible  and  include  some  or  all 
of  the  following:  (1 )  identify  those  to  whom  the  policy  applies,  (2)  identify  functions  that  are 
considered  acceptable  for  contracting  out,  (3)  cite  safeguards  to  be  used,  (4)  identify 
sanctions  for  failure  to  comply  with  the  policy,  (5)  provide  conflict  of  interest  mitigation 
strategies,  (6)  establish  approvals  and  approval  levels,  (7)  emphasize  that  contractors  have 
no  decision  authority,  (8)  specify  that  contractors  cannot  commit  the  Government,  (9) 
suggest  best  practices  in  using  contractors,  (10)  identify  risks  and  how  to  manage  them,  (1 1 ) 
identify  what  contract  types  should  be  used,  (12)  require  non-disclosure  agreements  and 
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financial  disclosure  statements,  (13)  suggest  metrics  to  be  used  in  evaluating  contractor 
performance,  (14)  enumerate  required  contractor  credentials  or  qualifications,  and  (15) 
outline  the  extent  to  which  contractors  can  participate  in  Government  events  outside  the 
workplace. 

Conclusions 

Contracting  out  of  procurement  functions  has  been  effective,  however,  robust 
metrics  to  measure  and  assess  contractor  performance  are  lacking.  Almost  all 
respondents  stated  that  contracting  out  of  procurement  functions  was  effective,  however, 
most  utilized  mission  attainment  and  perceptions  of  overall  contractor  performance  as 
metrics.  These  measures  should  not  be  discounted,  as  they  are  clearly  important  to  the  end 
user.  However,  they  represent  a  fuzzy  account  of  effectiveness  without  clear  criteria.  There 
were  no  comprehensive  or  universal  metrics  nor  framework  utilized  for  determining 
effectiveness  across  process,  workforce,  and  outputs  with  regard  to  quantitative  measures 
(objective)  and  qualitative  measures  (subjective).  Any  specific  metrics  cited  were  generally 
being  utilized  in  an  ad  hoc  and  inconsistent  manner.  In  light  of  capacity  and  capability 
shortfalls,  the  ability  to  utilize  contractors  to  complete  essential  missions  is  considered  a 
success.  However,  long-range  assessment  of  effectiveness  against  established  criteria  is 
not  occurring. 

The  phrase  “inherently  governmental  function”  continues  to  be  inconsistently 
interpreted  and  applied  throughout  DOD.  The  blurred  distinctions  between  inherently 
governmental  and  non-inherently  governmental  functions  caused  by  the  discretionary  ability 
of  agencies  to  decide  its  borders  will  continue  to  trouble  the  acquisition  process  until 
clarification  has  occurred.  The  Acquisition  Advisory  Panel  has  recommended  that  OFPP 
update  the  principles  agencies  use  in  determining  which  functions  must  be  performed  by 
Government  employees.  This  study  has  served  to  confirm  that  such  a  recommendation  is 
valid  and  pressing. 

Personal  services  relationships  are  almost  inevitable  in  the  close  working 
circumstances  required  between  Government  contracting  employees  and  contractor 
personnel  performing  procurement  functions.  The  lines  between  the  buyer-seller 
relationship  and  the  employer-employee  relationship  have  become  more  distorted  than  ever. 
Contract  award  requires  much  higher-level  review  and  interaction  with  the  author  of 
documents.  These  exchanges,  if  done  between  Government  Contracting  Officers  and 
contractors  could  be  interpreted  as  personal  services.  The  co-location  of  contractor 
employees  in  Government  facilities  certainly  creates  the  appearance  that  they  are 
Government  employees  if  not  actual  treatment  as  such.  The  contracting  functions  that 
might  be  contracted  out  are  frequently  so  closely  intertwined  with  functions  that  must  be 
performed  by  Government  personnel  that  a  personal  services  relationship  will  almost 
certainly  develop. 

The  contracting  community  is  seriously  concerned  about  the  potential  for 
conflicts  of  interest,  both  organizational  and  personal,  when  contractors  are  used  to 
perform  contracting  functions.  The  issue  of  conflicts  of  interest  has  come  up  on 
numerous  occasions  during  this  research.  Legal  concerns  almost  always  turn  into  a 
discussion  of  conflicts  of  interest  and  ethical  considerations  frequently  result  in  the  same 
scenario.  Government  employees  have  been  so  carefully  trained  over  the  last  several  years 
regarding  not  only  inappropriate  but  also  illegal  behavior  that  they  are  very  sensitive  to  not 
only  actual  situations  but  also  the  perception  of  conflicts.  One  step  to  lessen  the  potential 


ACQUISITION  RESEARCH:  CREATING  SYNERGY  FOR  INFORMED  CHANGE  -  421  - 


for  conflicts  of  interest  would  be  to  establish  firewalls  within  the  organization  that  prevent 
contractor  employees  from  operating  outside  the  specific  boundaries  of  their  particular 
project.  In  reality,  this  becomes  very  expensive  to  structure  and  enforce.  One  drawback  is 
that  this  does  not  allow  contractor  personnel  to  transfer  knowledge  and  freely  interact  with 
Government  contract  specialists  outside  their  firewall,  which  has  been  cited  as  a  benefit  to 
having  contractors  present  in  the  first  place.  The  Acquisition  Advisory  Panel  believes  that 
because  the  FAR  provides  considerable  leeway  to  agencies  in  addressing  actual  or 
potential  conflicts  of  interest  and  because  there  is  a  lack  of  guidance  in  mitigating  such 
conflicts  leading  to  inconsistent  application  of  the  regulations,  uniform  regulatory  language  is 
needed. 

Specific  measures  must  be  taken  to  ensure  ethical  standards  are  maintained 
and  the  integrity  of  the  contracting  process  is  protected.  One  might  assume  that  so 
much  education  has  gone  into  shaping  the  ethical  character  and  identity  of  the  acquisition 
workforce  in  recent  years,  that  this  conclusion  would  be  unnecessary.  The  Darleen  Druyun 
affair  caused  considerable  consternation  within  the  acquisition  workforce,  particularly 
contracting  community.  This  situation  came  up  during  interviews  in  the  context  of  ethical 
principles  and  an  imperative  need  to  protect  the  integrity  of  the  acquisition  and  contracting 
processes.  It  caused  acquisition  organizations  to  “pull  back”  and  reexamine  the 
fundamental  structure  of  their  ethical  climate.  Are  the  right  checks  and  balances  in  place  to 
prevent  or  discourage  such  events?  Does  the  senior  leadership  put  correct  and  suitable 
emphasis  on  ethical  principles  and  moral  values?  Are  instances  of  ethical  and  standards  of 
conduct  transgressions  handled  in  a  vigorous  fashion?  Are  subtle  indiscretions  and 
instances  of  wrongdoing  dealt  with  promptly  and  aptly?  Have  we  assured  industry,  with 
overt  measures,  that  such  irresponsible  actions  will  not  be  repeated?  A  majority  of  survey 
respondents  deem  that  ethical  issues  are  clearly  associated  with  contracting  for 
procurement  services  and  that  specific  actions  are  necessary  to  protect  the  integrity  of  the 
contracting  process. 

Contracting  out  of  procurement  services  will  have  a  negative  affect  on  the 
ability  of  the  Federal  Government  to  develop  Contracting  Officers  but  may  expand  the 
ability  of  Government  agencies  to  develop  procurement  options.  Over  65%  of  the 

respondents  maintained  the  former.  Contracting  Officers  grow  from  the  experience  of  having 
worked  as  contract  specialists.  They  need  to  grasp  the  underlying  fundamentals,  concepts 
and  basics;  understand  the  implications  of  particular  courses  of  action;  and  experience 
various  assignments  that  require  critical  thinking,  interpretation  of  regulations  and  policies, 
judgment  skills,  cultural  awareness  and  the  ability  to  make  trades.  All  of  these  skills  and 
abilities  are  acquired  on  the  job,  interspersed  with  appropriate  levels  of  training  and 
education.  As  the  decision  is  made  to  increase  the  number  of  contractors  in  the  workforce, 
there  will  be  fewer  full  time  civil  servants  hired,  which  ultimately  reduces  the  pool  of  potential 
Contracting  Officers.  If  lower  level  functions  are  contracted  out,  they  may  be  ill-prepared  to 
do  the  more  complex  tasks  later  in  their  careers.  Over  40%  of  the  respondents  felt 
procurement  options  would  be  expanded.  They  cited  that  contractors  can  bring  new  ideas 
to  the  discussion,  they  can  interject  industry  business  methods,  they  allow  Government 
personnel  to  perform  more  difficult  tasks,  they  can  come  with  specific  skills  and  expertise, 
and  they  may  have  innovative  approaches  which  can  be  shared  with  the  Government. 
Government  procurement  has  long  been  reproached  for  its  lack  of  understanding  of 
commercial  methods  and  sound  business  concepts  and  practices.  Contractors  may,  for 
example,  be  able  to  accomplish  and  supply  market  research  and  planning  alternatives  that 
would  be  more  difficult  for  Government  personnel  to  provide. 
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There  is  mixed  opinion  regarding  the  affect  contracting  out  of  procurement 
services  would  have  on  companies  participating  in  the  marketplace  for  Government 
contracts.  This  research  started  with  the  premise  that  there  could  be  a  negative  affect  on 
the  willingness  of  some  companies  to  either  compete  for  Government  contracts  or  to  be 
open  with  information,  technology  and  data  if  contractors  were  performing  Government 
contracting  functions.  If  companies  become  suspicious  of  their  treatment  in  Government 
competitions,  their  eagerness  to  continue  may  be  dampened.  Firms  are  more  likely  to 
question  the  integrity  of  the  process  if  non-Government  personnel  handle  proprietary 
information  and  participate  in  or  influence  acquisition  strategies  and  source  selections. 
Vendors  could  lose  confidence  in  the  fairness  and  objective  treatment  of  offerors.  Although 
a  majority  held  this  view,  several  respondents,  with  many  years  of  experience  in  using 
contracted  procurement  services  believed  that  there  would  not  be  a  negative  affect  on 
market  participation.  In  all  of  their  dealings  with  industry,  they  have  not  seen  any  perceived 
or  real  impact  on  market  participants.  They  felt  that  offerors  are  very  willing  to  do  business 
with  the  Government  even  though  other  companies  are  performing  contracting  functions. 

Government  contracting  functions  are  being  performed  by  contractors 
because  buying  organizations  lack  sufficient  human  resources  to  accomplish 
mission  requirements.  Time  and  again  throughout  the  surveys  and  interviews,  the  chief 
response  to  a  question  about  the  need  to  use  contractors  was  that  the  levels  of  Government 
personnel  are  too  low  to  permit  adequate  performance  of  the  workload.  Contracting  Officers 
and  contract  specialists  are  overwhelmed  and  feel  they  are  working  in  sweat  shops.  In 
certain  geographical  areas,  the  same  positions  in  other  Federal  agencies  are  far  less 
demanding  and  provide  the  same  level  of  compensation.  Openings  in  these  agencies  are 
very  attractive.  It  is  literally  impossible  to  fill  vacant  positions  with  qualified  applicants. 
Downsizing  actions  taken  in  the  acquisition  workforce  over  the  last  several  years,  large 
numbers  of  retirements,  hiring  freezes,  slowly  developing  intern  programs  and  a 
cumbersome  personnel  recruitment  system  all  have  added  to  the  problem.  Although  hiring 
authority  has  improved  in  very  recent  years,  there  is  a  significant  gap  between  the  entry 
level  and  the  journeyman  level  employee.  All  of  these  conditions  have  caused  some 
supervisors  to  solve  their  human  capital  dilemma  by  contracting  out. 

A  majority  of  senior  contracting  personnel  believe  that  contracts  for 
procurement  services  should  be  of  a  temporary  nature.  Over  60%  of  senior  contracting 
personnel  believe  it  should  be  on  a  temporary  basis  and  limited  to  surge  or  emergent 
demands  while  the  Government  recruits  and  trains  organic  resources.  There  is  recognition 
that  a  periodic  reevaluation  of  need  and  internal  capability  should  be  the  deciding  point  to 
continue  under  contract.  Although  these  contracts  are  viewed  as  “temporary,”  if  the  long¬ 
term  plan  is  to  convert  back  to  the  Government  any  functions  contracted  out  after  sufficient 
staff  has  been  recruited  and  hired,  then  the  timeframes  could  be  in  years.  If  the  contracts 
are  for  certain  functions,  such  as  market  research  or  requirements  development,  in  which 
the  contractor  is  typically  engaged  in  a  specific  acquisition  under  a  task  order,  then  these 
are  of  a  more  temporary  nature.  There  is  not  a  unanimous  opinion  that  they  be  temporary. 
Some  felt  they  were  recognizing  reality  by  pointing  out  the  long-term  problem  of  getting 
additional  Government  billets,  and  that  contracts  should  be  placed  on  a  permanent  basis 
until  and  unless  a  cadre  of  trained  Government  1102s  is  in  place,  which  will  take  years. 
Contract  closeout  is  an  example  of  a  recurring  need  in  which  organizations  may  never  be 
caught  up  and  contracted  services  are,  out  of  necessity,  integrated  into  the  normal 
workload. 


ACQUISITION  RESEARCH:  CREATING  SYNERGY  FOR  INFORMED  CHANGE  -  423  - 


Contractor  personnel  performing  procurement  functions  should  be  co-located 
with  Government  contracting  personnel.  There  is  a  need  for  close  communications  on  a 
face-to-face  basis  between  all  members  of  the  acquisition  team.  Contractor  personnel  must 
be  an  active  part  of  the  Government  team,  building  solid  working  relationships  and  learning 
from  each  other.  Physically  separating  Government  and  contractor  employees  hampers 
communication  and  would  not  create  a  very  conducive  work  environment  or  atmosphere 
and  might  tend  to  develop  an  “us”  versus  “them”  mentality.  With  multi-functional 
Government  teams,  it  could  be  detrimental  to  segregate  out  the  contractor  employees. 
Services  involve  personal  interaction  and  relationships.  Physical  separation  simply 
artificially  complicates  performance  of  a  cohesive  objective.  The  interface  that  occurs 
through  physical  proximity  outweighs  most  risks  that  might  surface.  Professional  interaction 
and  synergy  are  needed  to  efficiently  perform  the  functions.  There  is  a  need  to  be  close  to 
the  customer  for  effective  support  and  to  reap  the  efficiencies  of  real-time  decision-making. 
Also,  co-location  will  aid  in  the  performance  of  the  Government’s  responsibility  for  contractor 
oversight.  Examples  exist  in  which  contractor  employees  were  physically  separated  but 
were  relocated  to  the  Government  facility  because  the  ability  of  Government  personnel  to 
interact  with  contractors  was  very  difficult.  It  is  true  that  there  is  the  potential  for  direction  by 
the  Government  leading  to  personal  services,  and  physical  separation  would  assist  in  the 
perception  that  the  services  are  not  personal.  Additionally,  access  to  sensitive  and 
proprietary  data,  security  considerations,  and  the  potential  for  conflicts  of  interest  is  of  real 
concern.  Safeguards  and  security  measures  must  be  taken  to  protect  against  such 
occurrences. 

The  percentage  of  the  contracting  workforce  and/or  the  percentage  of  the 
contracting  workload  placed  on  contract  for  performance  by  contractor  employees 
should  not  exceed  an  established  maximum.  This  study  evaluated  a  limitation  in  terms 
of  workforce  and  workload.  The  surveys  indicate  that  most  individuals  advocate  a  maximum 
somewhere  between  25%  and  50%  of  either  measure  of  effort.  Although  full  time 
equivalents  are  easy  to  measure  and  a  percentage  of  an  activity’s  end  strength  is  easy  to 
calculate,  it  is  the  view  of  this  research  that  the  percentage  limitation  be  applied  to  workload 
as  opposed  to  workforce.  Individual  contracting  members  of  a  buying  organization  perform 
a  range  of  tasks  and  duties.  Cutting  them  out  of  the  organization  slices  through  these  tasks 
without  regard  to  the  complexity  or  nature  of  the  tasks  involved.  Further,  when  focusing  on 
workload,  an  activity  can  group  various  tasks  that  are  candidates  for  performance  by 
contractors,  such  as  contract  closeout  duties,  and  apply  the  percentage  to  the  grouped 
tasks.  Government  contract  specialists  do  not  perform  just  one  set  of  duties,  such  as 
contract  closeout,  but  are  typically  engaged  in  a  fuller  range  of  responsibilities.  Workload 
considerations  also  permits  the  organization  to  think  in  terms  of  grouped  tasks  that  can 
easily  be  described  in  statements  of  work,  are  fairly  homogeneous,  may  be  of  low  risk,  and 
might  be  easily  severable  and  require  far  less  interaction  with  Government  personnel, 
thereby  potentially  even  allowing  performance  at  the  contractor’s  facility. 

Requirements  similar  to  those  found  in  the  Defense  Acquisition  Workforce 
Improvement  Act  (DAWIA)  could  be  imposed  on  contractor  employees  performing 
Government  procurement  functions  without  difficulty.  Sixty-five  percent  of  the  senior 
contracts  leadership  believed  that  DAWIA  or  DAWIA-like  certification  requirements  should 
be  imposed  on  contractor  personnel  performing  contracting  functions  for  the  Government. 
There  is  a  feeling  that  these  requirements  are  critical  elements  in  the  performance  of 
complex  functions  and  that  contractors  should  be  required  to  have  the  same  level  of 
competence  as  Government  personnel.  Contracting  tasks  have  become  more  complicated, 
of  a  high-risk  nature,  and  demand  intellectually  capable  personnel  who  can  reason  through 
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the  issues  with  common  sense  and  wisdom.  Many  of  the  contractor  personnel  working  in 
Government  offices  are  former  civil  servants  or  military  who  achieved  DAWIA  certification 
while  in  the  Government.  They  most  likely  already  hold  the  requisite  credentials.  If  serious 
thought  is  given  to  requiring  DAWIA,  the  extent  to  which  standards  already  exist  that  are 
comparable  to  DAWIA-type  requirements  should  be  explored.  Professional  association 
certification  programs,  industry  association  and  corporate  training  programs,  and  academic 
certificate  programs  are  all  examples  of  existing  or  potential  methods  for  alternatively 
meeting  DAWIA-type  standards. 

There  is  general  opposition  among  Navy  and  Marine  Corps  contracts  leaders 
to  the  notion  of  contracting  out  procurement  functions.  Through  interviews  and  from 
the  surveys,  it  has  become  apparent  that  most  of  the  Navy  and  Marine  Corps  contracting 
leadership  are  generally  opposed  to  contracting  out  procurement  functions.  Although  they 
will  acknowledge  that  some  contracting  functions,  such  as  contract  closeout,  are  being 
effectively  performed  by  contractors  and  might  not  otherwise  be  accomplished  in  a  timely 
fashion,  the  vast  majority  of  tasks  are,  in  their  estimation,  so  closely  intertwined  with 
inherently  governmental  functions  that  they  must  be  performed  by  Government  civil 
servants.  Even  in  those  cases  in  which  a  buying  organization  is  utilizing  contractors  fairly 
extensively,  there  is  a  feeling  that  if  sufficient  qualified  personnel  were  available  they  would 
rather  accomplish  all  mission  requirements  with  Government  employees. 

A  policy  regarding  the  contracting  out  of  procurement  functions  is  needed. 

Sixty  percent  of  the  senior  leadership  believed  a  policy  is  needed  to  set  the  general 
boundaries  for  contracting  out  and  would  be  extremely  helpful  in  clarifying  top 
management’s  position  on  all  aspects  of  this  issue.  Their  main  concern  is  that  there  is  too 
much  difference  of  opinion  concerning  the  definition  of  inherently  governmental  functions 
and  a  more  direct  application  to  contracting  functions  is  needed.  This  is  coupled  with  the 
perceived  need  for  an  identification  of  conflict  of  interest  mitigation  strategies,  metrics  to  be 
used  in  evaluating  contractor  performance,  appropriate  sanctions  for  contractor 
transgressions,  the  qualifications  and  credentials  that  should  be  required  of  contractors,  the 
hidden  risks  involved,  and  best  practices  in  using  contractors,  to  name  just  a  few. 

Recommendations 

Metrics  should  be  developed  and  robustly  utilized  to  monitor  and  assess 
contractor  performance  of  Government  contracting  functions.  This  research  has  found 
that  the  acquisition  workforce  believes  that  the  procurement  of  contracting  functions  have 
been  relatively  effective  based  on  only  two  primary  factors:  (1)  was  the  mission 
accomplished  in  that  the  contracting  functions  were  performed,  and  (2)  did  the  contractor 
perform  well  enough  to  be  considered  for  future  contracts.  As  was  discussed  earlier,  there 
are  obvious  shortfalls  in  the  existing  means  to  determine  effectiveness,  in  that  it  lacks  clearly 
defined  criteria  and  the  degree  to  which  the  value  of  effectiveness  is  determined.  To  aid  in 
the  development  of  valid  metrics  to  determine  effectiveness,  it  is  recommended  that 
organizations  use  the  model  depicted  in  Appendix  1 .  This  study  suggests  specifics  that 
could  be  used  as  a  starting  point  for  creation  of  organization-unique  metrics  and  measures 
for  local  application  that  can  be  inserted  into  the  appropriate  sections  of  the  model.  Data 
requirements  should  be  tailored  within  the  model  framework  for  each  unique  application  and 
should  strike  a  proper  balance  to  ensure  that  they  can  elicit  contractor  performance 
consistent  with  strategic  organizational  performance  goals.  Activities  should  utilize  this 
model  framework  as  the  basis  of  construct  for  any  contracting  action  for  procurement 
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functions.  Higher  level  management  and  policy  personnel  can  use  this  model  to  gather  and 
disseminate  informational  and  actionable  metrics  within  their  organizations.  The  value  of 
this  model  is  that  it  brings  together  the  quantitative  (objective)  and  qualitative  (subjective) 
dimension  together  with  the  three  types  of  metric  categories  (process,  workforce,  and 
outputs)  and  overlays  these  on  the  six  phases  of  the  contracting  process.  The  contractor’s 
work  effort  must  be  evaluated  and  assessed.  Government  contracting  personnel  will  have 
the  principal  responsibility  for  performing  these  evaluations.  In  actuality,  assessing  the 
performance  of  a  contractor  performing  Government  contracting  functions  is  not  unlike  the 
assessment  that  must  occur  when  Government  managers  and  supervisors  are  evaluating 
their  own  civil  servant  workforce.  Many  of  these  judgments  are  highly  subjective  in  nature 
but  nonetheless  must  be  performed.  It  is  suggested  that  this  model  will  assist  in  that  difficult 


task. 


The  Department  of  Defense  should  issue  a  policy  regarding  the  contracting  out 
of  procurement  functions.  There  is  overwhelming  evidence,  as  brought  out  in  surveys 
and  interviews,  that  some  type  of  policy  should  be  disseminated  from  the  Office  of  the 
Secretary  of  Defense  that  will  guide  the  Services  and  Defense  Agencies  through  the 
challenges  created  by  utilizing  contractors  to  perform  Government  contracting  functions. 

This  is  not  to  say  that  organizations  have  not  already  successfully  carried  out  the 
responsibility  of  awarding  and  administering  contracts  under  which  contractors  are 
effectively  and  productively  executing  these  duties.  At  a  minimum,  the  policy  should 
address  the  areas  brought  out  earlier  in  this  paper.  Such  a  policy  will  go  a  long  way  in 
helping  to  clarify  many  of  the  issues  currently  plaguing  DOD  acquisition  organizations. 

Safeguards  to  protect  the  integrity  of  the  contracting  process  when  using 
contractor  support  to  accomplish  contracting  functions  should  be  strengthened  and 
rigorously  enforced.  Utilization  of  contractor  employees  to  perform  Government 
contracting  functions  is  relatively  new  and  has  posed  a  new  set  of  complex  challenges.  This 
research  has  confirmed  the  existence  of  serious  challenges  to  the  procurement  process  that 
have  already  been  known  on  a  fairly  widespread  basis.  Not  the  least  of  these  is  conflicts  of 
interest,  both  organizational  and  personal.  Other  challenges  that  may  increase  as  more  and 
more  contractors  become  involved  in  performing  contracting  actions  are  ethical  problems, 
personal  services  issues,  legal  issues,  and  general  overall  threats  to  the  integrity  of  the 
contracting  process.  Several  measures  are  already  in  place  to  protect  the  Government  from 
improper  and  unethical  behavior  on  the  part  of  both  civil  servants  and  contractors.  In  many 
cases,  however,  experience  has  shown  that  these  have  not  been  enforced  with  the 
thoroughness  and  as  meticulously  as  they  should  be  imposed.  Contracting  personnel 
interviewed  and  surveyed  for  this  research  have  repeatedly  pointed  to  the  potential  for 
biased  and  less-than-objective  action  on  the  part  of  contractor  employees  whose  loyalties 
and  motivations  may,  from  time  to  time,  be  at  odds  with  the  best  interests  of  the 
Government.  The  public  image  and  reputation  of  the  procurement  process  is  vital.  The 
“fishbowl”  environment  within  which  this  process  takes  place  sets  an  even  greater 
responsibility  for  preserving  an  untarnished  image  on  all  members  of  the  acquisition 
workforce. 

The  prohibition  on  the  use  of  personal  services  contracts  should  be  removed. 

Throughout  this  study,  references  to  the  difficulties  encountered  by  Government 
organizations  attempting  to  avoid  personal  services  situations  have  continually  arisen. 
Although  a  contract  may  have  been  carefully  crafted  to  eliminate  any  potential  for  such  a 
relationship,  including  a  precisely  defined  statement  of  work,  actual  contract  execution  may 
be  riddled  with  instances  where  the  line  has  been  crossed.  Some  organizations  have  taken 
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extreme  measures,  at  some  expense,  to  structure  working  relationships  that  meticulously 
avert  any  opportunity  for  personal  services.  Other  organizations  have  essentially  ignored 
the  rules  because  they  are  too  unrealistic  and  unworkable.  It  has  been  demonstrated  in  this 
study  that  the  close  working  relationship  so  important  to  effective  execution  of  contracting 
duties  requires  a  significant  amount  of  interaction  and  direction  that  is  of  a  personal  services 
nature.  This  recommendation  is  consistent  with  a  recommendation  by  the  Acquisition 
Advisory  Panel,  which  called  for  removal  of  the  restriction  regarding  supervision  of 
contractor  employees  by  Government  personnel.  All  of  the  other  aspects  of  the  employer- 
employee  relationship,  such  as  hiring,  firing,  performance  appraisal,  compensation, 
promotion,  etc.,  remain  exclusively  within  the  contractor’s  area  of  responsibility. 

Civil  and  criminal  penalties  currently  applicable  to  Federal  employees  should 
be  extended  to  contractor  employees  who  are  performing  contracting  functions  for 
the  Government.  It  has  been  noted  in  this  study  that  contractor  personnel  performing 
procurement  functions  on  behalf  of  the  Government  are  not  subject  to  the  same  penalties 
and  consequences  that  would  be  enforced  upon  civil  servants  for  violations  of  statutes, 
standards  of  conflict  and  ethical  principles.  The  Government  is  potentially  at  significant  risk 
for  unlawful  or  dishonest  actions  taken  by  contractor  employees  acting  on  its  behalf. 
Contractor  employees  are  not  liable  for  the  work  they  perform  or  the  recommendations  they 
make.  Sanctions  do  exist  for  illegal  or  improper  contractor  behavior,  such  as  suspension  or 
debarment,  but  this  generally  fails  to  recognize  employee  misbehavior.  Companies  that 
may  have  been  injured  by  the  unauthorized  and  prohibited  actions  of  a  contractor  employee 
performing  contracting  actions  may  have  recourse  against  the  Government  to  obtain  a 
remedy  for  an  offense,  but  the  Government,  in  turn,  does  not  currently  have  recourse 
against  that  employee  except  to  complain  to  the  contractor  and  seek  removal.  It  has  been 
expressed  by  many  that  contractor  personnel  performing  Government  contracting  actions 
should  be  held  to  the  same  standards  and  consequences  for  wrongdoing  as  civil  servants. 
This  could  occur  by  extending  appropriate  civil  and  criminal  penalties  to  those  contractor 
individuals  performing  contracting  functions. 

A  hierarchy  of  contracting  functions  should  be  developed  as  a  classification  of 
tasks  that  can  be  used  to  support  various  decisions  and  reporting  requirements. 

Appendix  2  presents  a  proposed  “Hierarchy  of  Contracting  Functions”  as  a  conceptual 
method  of  arraying  and  evaluating  contracting  tasks  or  functions  that  are  typically  performed 
by  Government  buying  organizations.  The  hierarchy  can  be  uniquely  tailored  to  each 
organization  to  reflect  their  specific  duties  at  the  micro  level.  The  hierarchy  can  also  be 
used  by  DOD  and  the  Services/Defense  Agencies  as  a  macro-level  approach  to 
categorizing  and  distinguishing  tasks  and  duties  by  specific  characteristics.  This 
taxonomical  approach  to  classifying  functions  permits  organizations  to  identify 
characteristics  that  differentiate  functions  from  one  another.  Once  an  organization  has 
defined  the  objectives  of  its  classification,  e.g.,  candidates  for  contracting  out,  individual 
tasks  can  be  placed  in  the  hierarchy  according  to  the  interpretation  it  has  made  about  each 
task.  Explicit  justification  for  the  category  of  placement  should  be  maintained.  The  hierarchy 
can  then  be  used  by  an  organization  as  an  inventory  of  functions  for  a  variety  of  purposes. 
One  such  purpose  would  be  to  support  submission  of  function  designations  under  the  FAIR 
Act.  The  hierarchy  could  also  be  used  by  organizations  as  a  common  framework  to 
compare  the  classification  of  tasks  and  supporting  rationale  with  each  other.  Additionally, 
capability  gaps  in  skill  levels  identified  by  the  DOD  Competency  Model  could  be  overlaid  on 
this  hierarchical  model  to  determine  where  a  particular  function  under  examination  resides  in 
the  overall  classification  scheme. 
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Appendix  2.  Hierarchy  of  Contracting  Functions 
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1.  Introduction 

“Because  they  remain  uncategorized  by  any  formal  measure,  the  exact  number  of 
PMFs  [private  military  firms]  that  have  entered  the  market  is  difficult  to  establish  and 
it  most  definitely  remains  in  constant  flux.  The  global  number  is  estimated  to  be  in 
the  mid-hundreds.”  (Singer,  2003,  p.  79) 

The  purpose  of  this  research  is  to  help  the  US  Department  of  Defense  and  other 
government  security  communities  better  understand  the  evolving  PM  (private  military)  sector 
(Avant,  2005;  Jager  &  Kummel,  2007;  Singer,  2003).  We  anticipate  our  readers  will  be 
government  agents,  members  of  the  international  community,  or  others  who  wish  to  make 
informed  decisions  regarding  the  use  of  PMFs.  This  report  discusses  a  beginning  step  in 
establishing  a  long-term  program  of  research  on  the  PM  sector  at  NPS.  It  is  expected  that 
knowledge  about  the  sector  will  be  built  incrementally  through  a  series  of  individual  studies; 
no  one  study  will  provide  a  complete  picture  of  the  relevant  features  of  the  sector.  However, 
we  believe  a  good  starting  point  is  to  develop  quantitative  data  about  the  industry,  which  this 
report  attempts  to  do.  We  suggest  three  reasons  why  this  is  an  appropriate  place  to  begin. 

First,  while  there  are  several  conceptual  and  qualitative  publications  on  the  sector, 
there  are  very  few  quantitative  studies.  Therefore,  there  is  an  important  gap  in  our 
knowledge  that  we  propose  to  fill  with  rigorous  quantitative  data. 

Second,  without  even  rudimentary  quantitative  data,  we  have  no  way  of  knowing 
whether  the  firms  that  have  been  studied  as  individual  cases  (such  as  MPRI  and  Executive 
Outcomes)  or  those  firms  that  have  been  given  significant  media  attention  (such  as 
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Blackwater  and  Halliburton)  are  typical  industry  participants  or  outliers.  Thus,  our  starting 
point  is  to  ensure  that  we  have  a  reasonably  accurate  picture  of  the  sector  by  compiling 
quantitative  data  on  it.  We  need  rudimentary  data  about  the  industry’s  size  in  aggregate 
(i.e.,  the  demography  of  firms — cf.  Singer’s  quote  above),  which  major  public  corporations 
(such  as  the  prime  US  defense  contractors)  are  active  in  the  sector  and  in  what  capacity, 
and  the  major  capabilities  of  firms.  Much  of  the  data  presented  in  this  report  fulfills  these 
needs.  The  data  is  basic,  but  since  our  collective  understanding  of  the  sector  is  also  fairly 
basic,  this  information  may  well  be  of  some  utility  to  readers  of  this  report.  As  well  as  giving 
us  a  sense  of  what  we  think  we  already  know,  the  data  also  provides  a  platform  on  which 
further  studies  can  be  built;  i.e.,  it  provides  a  context  in  which  future  research  can  be  set.  To 
proceed  to  these  more  advanced  topics,  we  must  first  pass  through  the  entry  gate;  that  is, 
we  must  ensure  that  the  basic  building  blocks  for  comprehension  and  analysis  are  in  place. 
We  think  that  at  least  some  of  the  data  we  exhibit  is  new  and  has — as  far  as  we  know — 
never  been  presented  before. 

Third — and  perhaps  most  importantly — while  most  of  our  data  lends  support 
to  already-published  literature,  the  process  of  studying  a  large  data  set  and 
attempting  to  analyze  it  has  brought  to  light  certain  discrepancies,  inconsistencies 
and  anomalies  between  the  way  the  sector  is  sometimes  described  and  the  reality  of 
the  empirical  data.  This  has  led  us  to  attempt  our  own  re-description  of  the  sector  in 
a  way  we  believe  is  more  congenial  to  the  data  we  have  collected. 

2.  Methodology  Used  and  Background  to  this  Study 

The  data  used  in  this  report  was  collected  primarily  by  three  NPS  MBA  students 
(Jared  Mitchell,  Don  Robbins  and  Chuck  Dunar)  working  on  their  thesis  project  in  the  fall  of 
2007  under  the  supervision  of  Nick  Dew  and  Bryan  Hudgens.  The  combined  faculty  and 
student  input  into  the  data  collection  effort  approximates  one  man-year  of  work. 

Data  collection  proceeded  through  three  phases.  We  started  by  assembling  a  list  of 
firms  known  to  be  active,  or  to  have  been  active  at  one  time,  in  the  PM  sector.  We  screened 
various  publications  about  the  industry  for  an  initial  list  of  PMFs  (for  example,  Avant,  2005; 
Singer,  2003).  Based  on  this  initial  list  of  names,  we  assembled  further  names  of  firms 
using  a  snowball  method  (Goodman,  1961);  i.e.,  our  searches  for  information  on  the  initial 
names  invariably  turned  up  new  firms,  which  we  then  added  to  the  list.  We  kept  working  on 
the  snowball  until  we  exhausted  the  search  for  new  names;  i.e.,  further  searches  did  not 
reveal  any  new  firms.  Almost  all  of  this  searching  was  conducted  online,  using  various 
databases  available  through  the  NPS  library  and  public  resources  available  via  online 
search  engines.  Using  this  methodology,  we  assembled  a  list  of  550  firms  “named”  by  one 
source  or  another  as  having  been  active  in  the  sector. 

Second,  we  found  that  many  firms  in  the  sector  have  a  website  which  offers 
information  about  the  organization.  Using  these  and  other  resources,  we  assembled  more 
detailed  data  on  the  firms  in  our  sample — such  as  their  founding  date,  founder  background 
information,  country  of  origin,  and  data  on  the  capabilities  these  firms  purport  to  have. 

Third,  one  of  us  conducted  follow-up  and  fill-in  data  gathering  on  specific  firms  as 
part  of  the  writing  of  this  report. 


ACQUISITION  RESEARCH:  CREATING  SYNERGY  FOR  INFORMED  CHANGE  -  431  - 


Further  elements  leading  to  the  assembly  of  this  report  included  coding  data  in  our 
database,  analyzing  the  raw  and  coded  data,  and  presenting  it  in  easily  understood  formats. 
Coding  (for  instance)  of  capabilities  was  conducted  by  two  of  us  (one  student,  one  author). 
This  process  was  particularly  lengthy  and  laborious,  since  it  involved  over  2,500  lines  of 
data  on  capabilities,  many  redundant  descriptions  of  capabilities,  and  much  recoding  work  in 
order  to  get  the  data  into  a  “clean”  format.  Individual  fields  were  coded  independently,  and 
critical  variables  (such  as  codings  of  Singer’s  “Tip  of  the  Spear”  schema  and  Avant’s 
categorization  scheme  for  contract  types)  were  coded  by  both  coders.  Though  we  have  not 
yet  measured  inter-rater  reliability  of  these  codings,  we  estimate  that  more  than  80%  of 
codings  are  identical. 

The  analysis  process  involved  several  iterations  in  order  to  produce  the  final  charts, 
graphs  and  data  presented  in  this  report.  The  final  portion  of  the  research  process  involved 
both  finding  ways  to  display  the  data  in  formats  that  are  easy  for  the  reader  to  understand 
and  writing  this  report. 

The  limitations  of  this  study  are  worth  particular  attention.  For  most  data  categories, 
the  data  on  PMFs  is  incomplete.  For  instance,  we  managed  to  find  data  on  the  founding 
dates  of  approximately  230  firms  (approximately  40%  of  our  sample).  We  obtained  data  (at 
least  in  some  rudimentary  form)  on  capabilities  for  approximately  70%  of  firms,  but  the 
quality  of  this  data  (measured  in  terms  of  its  comprehensiveness  and  trustworthiness)  varies 
considerably.  The  bottom  line  of  our  data-collection  effort  is  that  we  can  only  analyze  the 
data  available,  doing  our  best  to  verify  its  reasonableness  as  we  go.  We  cannot  attest  for 
the  accuracy  of  some  aspects  of  this  data — for  example,  that  the  capabilities  firms  purport  to 
have  are  “true.”  Of  course,  the  accuracy  of  self-reported  data  is  a  problem  for  researchers 
generally,  and  not  for  our  study  alone. 

However,  we  do  not  know  of  any  database  on  the  industry  that  is  more 
comprehensive  than  the  one  we  have  assembled.  As  far  as  we  know,  the  sample  size  we 
have  used  is  much  larger  than  any  other  so  far  studied  in  the  sector,  and  this  should  make 
our  results  more  robust  because  of  the  (generally)  favorable  statistical  properties  of  large 
samples. 

3.  Organizational  Demographics  of  the  PM  Sector 

Founding  Dates,  Population  and  Industry  Growth 

For  this  study,  we  traced  data  on  550  firms  that  appear  to  have  been  active  in  the 
PM  sector.  Of  these,  we  were  able  to  find  data  on  the  founding  dates  of  approximately  230 
firms.  Based  on  this  sample  of  230  firms,  the  following  pattern  of  industry  entry  emerges: 
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Figure  1.  Pattern  of  PMF  Founding  by  Year  1970-2006 

Further  examination  of  the  data  indicates  that  half  the  firms  for  which  we  managed  to 
find  founding  dates  were  founded  between  1995  and  2007;  the  other  half  were  founded 
before  1995.  This  makes  the  PM  sector  a  relatively  young  industry:  half  the  industry  is  less 
than  13  years  old;  thus,  the  median  age  of  firms  is  quite  low.  This  fact  is  an  interesting 
contrast  to  the  history  of  mercenary  companies,  which,  of  course,  has  very  deep  roots — 
stretching  back  at  least  until  the  Early  Modern  period  (15th  and  16th  Centuries)  (Oritz, 

2007a). 


Figure  2.  PMF  Founding  Pattern  by  Region 

Figure  2  suggests  that  the  recent  wave  of  entry  of  PMFs  is  predominantly  a  US 
effect.  Note  in  particular  the  trend  lines  for  firm  foundings:  the  US  trend  line  is  rather  steep, 
whereas  the  UK  and  ROW  (rest  of  world)  trend  lines  are  almost  flat.  What  this  trend 
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suggests  is  that  PMF  growth  is  being  driven  by  US  effects — such  as  outsourcing  strategy  in 
the  late  1990s  and  the  invasion  of  Iraq  in  2003. 

One  possible  way  of  thinking  about  entry  into  the  PM  sector  is  to  categorize  it  as 
occurring  in  different  “eras.”  Table  1  indicates  data  on  the  average  number  of  firms  founded 
in  three  different  eras. 

Table  1.  Summary  of  PMF  Foundings  in  Three  Different  “Eras” 


SUM 

YEARS 

AVG 

PRE-1970 

35 

25 

1 

1970-1989 

64 

20 

3 

1990-2001 

120 

12 

10 

2002-2006 

48 

5 

10 

If  we  examine  this  data  graphically,  we  observe  a  different  picture  of  the  trends  in 
PMF  founding  (note  the  trend  lines  are  generally  quite  flat  in  Figure  3).  Here,  we  can  see 
that  the  surge  in  PMFs  is  a  consistent  feature  of  the  post-Cold  War  era  that  is  driven  by  new 
US  firms  entering  the  sector.  This  means  that  the  sector  growth  is  not  a  post-9/1 1 
phenomenon  (as  is  commonly  thought),  but  a  post-Cold  War  phenomenon. 


Figure  3.  PMF  Foundings  1990-2006 
Geographic  Distribution  of  PMFs 

We  found  it  comparatively  simple  to  retrieve  data  on  the  nationality  of  PMFs.  We 
found  data  for  over  500  firms  (90%)  of  our  population.  Note  that  our  data  is  very  consistent 
with  the  IPOA’s  (International  Peace  Operations  Association)  surveys  (2006  and  2007)  that 
were  based  on  much  smaller  samples. 
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Figure  4.  Geographical  Distribution  of  PMFs 

As  is  evidenced  in  the  chart,  the  industry  is  a  50%  US  phenomenon  and  20%  UK. 
However,  this  was  probably  not  always  so.  The  tremendous  growth  in  the  number  of  US 
firms  entering  the  industry  in  the  past  10  years  (see  Figure  2)  has  probably  changed  the 
international  composition  of  the  industry.  Prior  to  the  burst  of  US  entry,  the  industry  was 
probably  more  cosmopolitan  in  its  composition. 

Several  factors  might  explain  this  international  distribution.  One  factor  might  be 
outsourcing  and  privatization,  which  may  make  PMFs  more  prevalent  in  the  US  and  UK. 
Another  factor  may  be  demand  factors — i.e.,  provision  of  surge  capacity  for  the  US  and  UK 
to  meet  their  commitments  in  Iraq,  Afghanistan  and  to  the  War  on  Terror.  A  further  factor 
may  come  from  the  supply  side — i.e.,  a  distinct  geographical  pattern  governing  the 
distribution  of  skills  required  for  establishing  PMFs.  This  requires  the  combination  of 
specialist  military/security  skills  and  generalist  entrepreneurial  skills.  These  may  be  more 
prevalent  in  the  US  and  UK.  Lastly,  we  have  not  yet  tested  this  distribution  for  correlation 
with  more  general  factors,  such  as  defense  spending  in  these  particular  geographies  (PMFs 
per  $BN  defense  spend),  population  (PMFs  per  million),  or  economic  scale  (PMFs  per  $BN 
GDP).  Future  research  might  endeavor  to  explore  these  and  other  possible  relationships. 
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Founder  Background  for  PMFs 


10% 


□  Commercial  &  other 

■  Former  intelligence  services 

□  Former  law  enforcement 

□  Former  military 

■  Former  spec  ops 


Figure  5.  PMF  Founder  Backgrounds 

(based  on  Data  on  1 16  firms) 

Figure  5  presents  the  data  we  managed  to  collect  on  founder  characteristics  for  1 16 
PMFs  (approximately  20%  of  our  sample).  What  this  reveals  is  initially  unsurprising:  most 
PMFs  are  founded  by  individual  entrepreneurs  or  entrepreneurial  teams  that  have  prior 
military  experience.  However,  the  proportion  of  firms  founded  by  individuals  with  special 
operations  experience  is  a  surprise:  almost  40%  of  firms  were  founded  by  individuals  with 
this  background. 

One  possible  explanation  for  this  result  is  that  our  sample  is  skewed:  perhaps  firms 
founded  by  individuals  with  special  operations  experience  are  more  likely  to  “tout”  their 
qualifications.  Other  possibilities  include  both  demand-side  and  supply-side  factors.  On  the 
demand  side,  perhaps  special  operations  skill  sets  are  in  particularly  high  demand  in  the  PM 
sector,  or  these  activities  are  seen  as  particularly  good  or  easy  targets  for  contracting-out. 

On  the  supply  side,  one  again  wonders  about  the  combination  of  skills  required  for 
running  a  successful  PMF:  perhaps  individuals  with  entrepreneurial  tendencies  are  more 
likely  to  select  into  special  operations  domains,  or  perhaps  special  operations  experience 
tends  to  nurture  particular  organizational  skills  and  self-confidence  that  lead  individuals  to 
participate  in  an  entrepreneurial  endeavor. 

Based  on  our  initial  analysis,  founder  characteristics  would  make  a  good  topic  for 
future  research  on  the  PM  sector. 
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■  Private  -  suspected 

□  Public 

□  Named  subsidiaries  of  Public  firms 


Figure  6.  Public-private:  PMF  Status 

Data  we  gathered  indicates  that  well  over  90%  of  the  firms  in  the  PM  sector  are 
privately  held.  Only  a  few  firms  (25)  active  in  the  sector  are  publicly  held.  This  number  is 
based  on  a  generous  definition  of  the  industry  and,  therefore,  includes  the  major  defense 
contractors  (the  “primes”)  and  many  firms  that  mainly  supply  IS/IT-related  products  and 
services  to  the  DoD  and  other  security  agencies  worldwide.  The  number  of  “pure  play” 
public  PMFs  is,  in  fact,  very  low:  only  two  firms  in  our  sample  meet  this  definition  (DynCorp 
and  ArmourGroup). 

One  important  issue  inherent  in  this  analysis  is  the  lack  of  transparency  in  the 
industry;  this  flaw  is  frequently  highlighted  by  critics.  PM-sector  firms  are  perceived  to  be 
rather  secretive  (Avant,  2005).  Our  data  points  to  the  fact  that  there  are  two  elements 
involved  in  this  secrecy: 

■  First,  private  firms  generally  lack  transparency  to  outsiders,  regardless  of  their 
industry.  Some  of  this  is  a  systematic  side-  effect  of  being  private,  not  the  result  of 
deliberate  policy.  After  all,  they  are  not  required  to  be  transparent,  and  they  have  no 
reason  to  be.  If  most  PMFs  are  private,  then  one  would  expect  the  industry  to  lack 
transparency  regardless  of  its  activity  type. 

■  Second,  PMFs  have  other  legitimacy-related  concerns  and  sometimes  security- 
related  reasons  for  shying  away  from  the  public  eye.  Thus,  their  privacy, 
opaqueness,  ambiguity  and  general  lack  of  transparency  may  be  a  deliberate 
strategy.  This  element  is  over-and-above  what  is  common  to  all  private  firms. 

A  second  concern  brought  to  light  by  our  data  is  that — despite  some  claims  to  the 
contrary  (for  instance,  Valero,  2008) — it  seems  rather  unlikely  that  the  PM  sector  will  ever 
emerge  as  a  significant  aspect  of  the  so-called  military-industrial  complex.  The  sector’s 
organization  is  quite  dissimilar  from  that  of  equipment  manufacturers;  unlike  the 
manufacturing  sector,  the  PM  sector  simply  has  not  the  economies  of  scale  that  have  driven 
a  concentration  of  large  players  (the  “primes”).  Instead,  the  industry  is  highly  dispersed — 
i.e.,  populated  by  firms  that  are  generally  quite  small  compared  to  the  defense-equipment 
sector. 
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4.  Capabilities/Activities  Analysis 

Our  database  contains  2,500  lines  of  data  on  the  capabilities/activities  of  395  PMFs. 
The  comprehensiveness  of  this  data  varies  by  firm;  but  as  a  starting  point,  we  believe  it  is  a 
useful  approximation  of  what  firms  in  the  sector  do.  To  help  analyze  the  data,  we  began 
with  the  categorizations  provided  in  the  literature  on  the  PM  sector,  i.e.,  Singer  (2003)  and 
Avant  (2005). 

A  Starting  Point:  Singer’s  Categorization  of  PMFs 

One  popular  device  that  emerged  from  Singer’s  (2003)  book  on  the  PM  sector  is  the 
’’tip  of  the  spear”  analysis.  Singer  used  this  tool  to  help  explain  the  industry  and  then  used 
case  studies  of  particular  firms  in  different  places  on  the  spear  to  illustrate  the  analysis  in 
more  depth.  For  instance,  he  posed  EO  (Executive  Outcomes)  as  the  quintessential 
“Military  Provider  Firm,”  MPRI  as  an  example  in  the  “Military  Consultant  Firm”  category,  and 
KBR  as  an  example  of  a  “Military  Support  Firm.”  See  the  figure  below  for  a  reproduction  of 
Singer’s  diagram: 


Implementation/ 

Command 


Advisory  and 
Training 


Non-lethal  Aid 
and  Assistance 


Figure  7.  Singer’s  “Tip  of  the  Spear” 

(Singer,  2003,  p.  93) 
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Avant’s  Refinements  to  Singer’s  Categorization  Scheme:  Form  Firms  to 
Contracts 

Deborah  Avant  (2005)  proposed  a  slightly  different  approach  to  the  analysis  Singer 
provided.  She  found  that  it  was  difficult  to  classify  individual  firms  using  Singer’s  typology 
because  many  firms  are  diversified,  offering  a  variety  of  services  that  appear  in  different 
places  on  the  spear  (e.g.,  Blackwater  does  close  protection,  firearms  training,  has  a 
parachuting  training  team  and  produces  an  armored  vehicle,  among  its  activities). 

Moreover,  Avant  found  that  firms  move  around  the  spear,  offering  different  services  to 
different  buyers  at  different  points  in  time.  For  these  reasons,  Avant  proposed  that  contracts 
are  a  better  tool  for  analyzing  the  sector.  She  categorized  contracts  according  to  five  types, 
as  follows: 


A 


DO 

> 


m 

to 

"0 

> 

o 

m 


* 


Military 


Armed 

Operational 

Support 


Unarmed 
Operational 
Support  on  the 
Battlefield 


Unarmed  Military 
Advice  and 
Training 


Logistical  Support 


Police 


Armed  Site  Security 


Unarmed  Site 
Security 


Police  Advice  and 
Training 


Crime  Prevention 


Intelligence 


Figure  8.  Avant’s  Analysis  of  the  Spear 


(Avant,  2005,  p.  17) 
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A  Further  Revised  “Tip  of  the  Spear”  Analysis:  From  Firms,  to  Contracts,  to 
Capabilities 

Singer’s  analysis  focused  on  firms;  Avant’s  analysis  focused  on  contracts.  In  what 
follows,  we  offer  an  extension  of  these  analyses  that  focuses  on  capabilities.  This  analysis 
is  premised  on  the  observation  that  firms  are  not  only  diversified  and  move  around  the 
spear,  but  that  an  analysis  of  what  tasks  firms  have  been  performing  over  time  does  not 
capture  firms’  potential  movement  around  the  spear.  There  is  an  even  broader  scope  of 
latent  activity.  One  way  to  investigate  this  latent  potential  is  to  collect  data  on  the 
capabilities  firms  claim  they  have.  The  following  section  focuses  on  these  capability  sets. 

Capabilities  are  critical  because  underlying  contracts  (transactions  in  the 
marketplace)  are  firm-level  capabilities.  The  concept  of  capabilities  is  widely  used  for 
analysis  in  the  strategic  management  literature  because  it  focuses  on  the  building  blocks  for 
activities  that  are  present  in  a  firm  (and,  therefore,  an  industry  sector).  Firms  distinguish 
themselves  by  their  capabilities — firms  are  able  to  get  contracts  others  cannot  access 
because  they  can  either  do  things  other  firms  cannot  or  can  do  them  at  a  lower  cost  than 
their  competitors  can.  Therefore,  in  strategic  management,  capabilities  are  often  thought  of 
as  crucial  underlying  variables  that  explain  the  relative  performance  of  firms  (Barney,  1991; 
Teece,  Pisano  &  Shuen,  1997). 

Based  on  our  attempts  to  cluster  the  approximately  2, 500  individual  capabilities  in 
our  data  set,  a  rather  different  image  of  the  “tip  of  the  spear”  emerged.  By  our  analysis,  the 
spear  is  much  more  heterogeneous  than  either  Singer  or  Avant’s  analysis  suggests.  The 
key  result  of  our  analysis  of  individual  capabilities  is  that  the  PM  sector  is  by  no  means 
unitary.  In  fact,  it  is  made  up  of  quite  different  sub-sectors,  which  are  probably  better 
thought  of  as  a  patchwork  quilt  than  as  elements  up  and  down  the  spear.  This  is  particularly 
true  for  the  category  “Military  support  firms,”  which  contains  a  smorgasbord  of  sub-sectors. 
These  sub-sectors  are  essentially  unrelated  to  one  another  in  terms  of  the  underlying 
capabilities  they  require  to  support  contracts  in  any  particular  area.  This  means  that  the 
firms  competing  for  contracts  in  these  sub-sectors  tend  to  come  from  very  different 
industries  (for  instance,  some  services  are  ’’add-ons”  provided  by  major  defense  contractors, 
while  other  services  are  provided  by  firms  with  capabilities  that  are  largely  undifferentiated 
from  civilian/commercial  skill  sets,  such  as  logistics  or  many  IS/IT  security  activities).  This 
led  us  to  present  a  revised  “tip  of  the  spear”  diagram,  displayed  in  Figure  9  below  (in  the 
figure,  the  individual  elements  are  not  sized  or  ordered  to  represent  the  data,  but  merely  to 
convey  an  overall  image  of  the  sector). 
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Figure  9.  Revised  “Tip  of  the  Spear” 


D.  Capability  Analysis 

Our  data  suggests  that  approximately  half  of  the  firms  in  our  sample  of  395  are 
engaged  in  some  kind  of  protective  and  security  services;  75%  do  advisory  and  training 
work,  and  almost  90%  are  engaged  in  some  kind  of  support  services  (variously  defined). 
This  data  points  clearly  to  the  intermingling  of  service  provisions  up  and  down  the  spear  that 
Avant  and  Singer  (and  others)  have  remarked  on  as  a  characteristic  of  the  sector. 
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protection  services 


Figure  10.  PMF  Activity  Summary 

(number  of  active  firms) 

For  the  exact  percentages,  see  the  table  below: 

Table  2.  Proportions  of  PMFs  Active  in  PM  Sub-sectors 

%  of  Firms  Active  (Sample  396) 


Protection  Services 

53% 

Advisory/Training 

76% 

Support  Services 

86% 

Another  way  to  display  this  data  is  as  proportions  of  the  spear — i.e.,  to  examine  the 
number  of  firms  active  in  different  sub-sectors  of  the  PM  sector: 
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Figure  11.  Number  of  PMFs  Active  in  Different  Sub-sectors 

Viewed  this  way,  about  25%  of  the  different  activities  occurring  in  the  sector  can  be 
classed  as  protective  services,  35%  as  training  and  advisory,  and  40%  (almost  half)  as 
support  services  of  various  kinds. 

Overall,  we  think  this  data  will  alarm  some  observers  and  satisfy  others.  Some 
people  will  be  alarmed  to  find  out  that  there  are  around  200  firms  offering  military 
competencies  of  various  kinds  for  sale  in  the  marketplace.  From  this  perspective,  it  is  rather 
worrisome  that  there  is  an  industry  that  specializes  in  fielding  various  kinds  of  (private)  mini¬ 
armies  to  the  highest  bidder.  Others  will  find  this  fact  reassuring  rather  than  worrisome — for 
them,  a  significant  number  of  firms  means  competition,  which  means  efficiency. 

Protective  and  Security  Services 

We  conducted  further  analysis  of  the  content  of  each  PM  sub-sector.  Results  for  the 
protective  services  segment  are  provided  in  Figure  12  below. 
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Figure  12.  PMFs  Engaged  in  Protective  Services 

(209  total) 

This  data  indicates  that  about  2/3  of  firms  who  are  active  in  the  provision  of 
protective  services  are  involved  in  close  protection  of  individuals  and  assets,  i.e.,  stationary 
guarding  and  convoy  protection.  When  an  individual  thinks  of  private  military  and  security 
contractors,  this  is  probably  what  comes  to  mind.  Our  data  indicates  that  this  role,  indeed,  is 
the  mainstay  of  the  protective  services  business. 

However,  there  are  other  protective  services  activities.  About  30  firms  are  known  to 
be  active  or  capable  of  providing  protective  services  for  marine  assets.  A  similar  number  of 
firms  have  capabilities  for  conducting  a  variety  of  operations.  The  kind  of  services 
mentioned  here  include  assault  capabilities,  rapid  reaction  forces,  and  special  operations 
units.  A  variety  of  miscellaneous  services  were  also  mentioned,  as  well  as  the  provision  of 
dog  teams  by  a  handful  of  firms. 

Training  and  Advisory  Services 

Most  firms  that  offer  advisory  (consulting)  services  also  offer  training  services.  There 
is  considerable  overlap  between  these  services,  as  indicated  in  Figure  13: 
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Figure  13.  PMFs  Engaged  in  Training  and  Advisory  Activities 

(240  firms) 

There  is  a  wide  range  of  advisory/consulting  services.  Commonly  mentioned 
advisory  services  are  risk/threat  analysis,  counter  terrorism  and  current  tactics.  However, 
we  found  that  a  very  diverse  range  of  advisory  capabilities  are  offered  in  the  marketplace. 
This  suggests  that — globally,  at  least — this  sector  is  quite  well-developed  and 
comprehensive  in  its  offerings. 

The  same  is  true  for  training;  our  data  indicates  a  very  diverse  range  of  training 
services  are  offered  by  firms — options  too  numerous  to  list.  According  to  our  data, 
approximately  200  firms  are  active  in  the  training  market  to  some  degree  or  another. 

Support  Services 

Based  on  our  earlier  analysis  displayed  in  Figure  9,  it  was  apparent  that  diversity  is 
also  a  hallmark  of  the  support  services  offered  in  the  PM  sector.  However,  some  services 
are  more  widely  available  than  others,  as  indicated  by  Figure  14  below: 
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Figure  14.  PMFs  Offering  Support  Services 

(data  from  335  firms) 

Two  pieces  of  data  seemingly  jump  out  of  Figure  14.  First,  consistent  with  Avant’s 
analysis  of  contracts,  intelligence  support  services  are  widely  available  in  the  sector.  This 
category  includes  a  range  of  services  such  as  surveillance,  intelligence  analysis,  various 
counter  measures,  and  information  gathering.  The  number  of  firms  active  in  this  service 
area  indicates  that  significant  competition  exists.  The  second  most  available  service  is 
IS/IT/Communications.  Again,  the  provision  of  these  services  appears  to  be  highly 
competitive,  with  many  firms  offering  a  diverse  range  of  activities  in  the  marketplace. 

Geographic  Distribution  of  Capabilities:  Do  Different  Geographies  Have 
Different  Capability  Sets? 

Are  some  geographies  “tippier”  than  others?  Do  some  geographies  have  a 
preponderance  of  consulting  or  support  services?  Based  on  our  data,  the  following  patterns 
emerged: 
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Figure  15.  Regional  Capabilities  Distribution 

(%  of  firms  offering  services  in  sub-sectors) 

When  examining  this  data,  we  must  remember  first  that  the  chart  shows 
percentages,  not  absolute  numbers  of  each  geographic  region’s  firms  active  in  each 
capability.  This  distinction  is  important  because  approximately  half  the  industry  is  based  in 
the  US,  and  this  would  otherwise  distort  the  data. 

The  pattern  that  emerges  from  Figure  15  is  that  US  firms  are  slightly  more  likely  to 
be  involved  in  support  services  and  slightly  less  likely  to  be  involved  in  protection  services. 
However,  overall,  there  is  little  difference  between  regions  when  the  service  mix  is  analyzed 
at  this  level.  Of  course,  the  service  mix  might  show  a  geographic  bias  in  narrower  capability 
segments.  We  have  not  yet  studied  this  data. 

Two  observations  might  be  worth  noting  when  we  investigate  the  data  on  capabilities 
this  way.  First,  a  focus  on  this  lower  unit  of  analysis  (i.e.,  a  lower  unit  of  analysis  than  whole 
firms)  offers  us  the  opportunity  to  examine  clusters  of  capabilities  (for  instance,  across 
different  geographies)  while  temporarily  ignoring  firms.  In  principle,  this  might  be  a 
reasonable  analytical  strategy;  it  is  well  known  that  most  firms  recruit  to  fill  contracts  from 
databases  on  individuals,  and  that  these  individuals  typically  appear  on  the  databases  of 
more  than  one  firm  (Singer,  2003).  Therefore,  what  might  be  important  is  the  availability  of 
these  individuals  and  their  capabilities  to  groups  of  firms,  rather  than  what  individual  firms 
do.  In  other  words,  firms  might  merely  be  “shells”  that  hide  underlying  capability  sets  that 
are  more  important  at  the  national  and  regional  level  than  at  the  firm  level. 

Second,  the  overall  similarity  of  the  industry  across  geographic  regions  points 
somewhat  to  the  international  nature  of  the  business.  While  there  is  significant  variation  in 
the  specific  offerings  of  individual  firms,  in  general  about  50%  of  firms  offer  protective 
services;  this  is  true  globally — regardless  of  a  firm’s  national  origins.  The  geographic 
proportions  hold  steady  for  training/advisory  services  and  support  services. 
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Reprise:  Defining  and  Bounding  the  PM  Sector 

Analysis  of  PMF  capabilities  invariably  leads  us  back  to  the  question  of  what,  and 
who,  belongs  inside  the  sector  (Oritz,  2007b).  In  conducting  our  survey,  we  initially  used  a 
generous  definition  of  firms  “active”  within  the  PM  sector  in  some  form  or  another.  But 
clearly,  the  definition  of  “sector”  and  “participant”  is  important  here.  Figure  16  summarizes 
the  various  ways  we  think  the  sector  might  be  defined: 


Protection 
&  Security 


Training 

& 

Advisory 


Support 

(Various) 


Figure  16.  Defining  the  PM  Sector 

In  this  diagram,  the  vertical  axis  explains  what  activity  is  taking  place.  The  activities 
are  arrayed  according  to  Singer’s  “tip  of  the  spear”  heuristic.  The  horizontal  axis  illustrates 
where  activities  are  occurring — either  in  hot  zones  (how  hot,  of  course,  varies)  or  at  home 
(i.e.,  some  other  safe  location). 

The  most  restrictive  definition  of  the  “industry”  would  focus  on  Segments  1  and  2  in 
this  figure.  The  2006  IPOA  survey  used  a  somewhat  restrictive  definition  of  the  sector — 
referring  to  firms  engaged  in  armed  security  operations,  which  yielded  a  sample  of  100 
firms.  In  its  report  on  PMFs,  Human  Rights  First  (2008,  p.  1)  used  a  similar  definition, 
explaining  that: 

There  is  no  universal,  agreed  definition  of  the  term  “private  security  contractor”  [...] 
Human  Rights  First  uses  here  an  essentially  functional  definition  of  the  term  in  light 
of  the  actual  activities  of  such  contractors  fielded  in  Iraq  and  Afghanistan  with  a  basic 
security  mission — that  is,  a  core  mission  to  protect  people  (other  than  themselves)  or 
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things,  to  include  guarding  government  (and  contractors)  facilities,  protecting 

government  personnel  (and  other  government  contractors)  and  United  Nations  (U.N.) 

and  other  international  organization  staff  as  well,  and  providing  security  for  convoys. 

Interestingly,  in  its  2007  follow-on  report  to  that  discussed  above,  the  IPOA 
broadened  its  survey  to  include  an  identified  sample  of  334  firms.  While  we  can’t  be 
completely  sure  of  the  Institute’s  criteria  for  inclusion  in  its  sample,  we  suspect  that  it  reflects 
Segments  1  and  2  in  the  diagram  above,  plus  Segments  3  and  4,  and  possibly  some  firms  in 
Segment  5. 

The  two  problem  zones  (or  “gray  areas”)  in  analyzing  the  PM  sector  are  Segments  5 
and  6.  Some  elements  of  Sector  5  fall  more  easily  inside  what  we  believe  most  analysts 
would  agree  as  defining  the  PM  sector.  For  instance,  Blackwater’s  North  Carolina  training 
range,  which  includes  various  weapons  ranges,  is  sometimes  touted  as  the  best  in  the  world 
for  some  types  of  military  training.  Many  aspects  of  MPRI’s  (Military  Professional 
Resources,  Inc.)  advisement  activities  would  also  clearly  fall  in  this  segment.  However, 
where  should  we  classify  activities  such  as  Cubic’s  virtual  training  systems?  Should  we 
include  firms  such  as  Cubic  in  the  PM  sector,  or  exclude  it?  We  think  there  are  arguments 
on  both  sides. 

Even  more  problematic  is  Segment  6  in  the  diagram.  There  are  two  issues  here. 
First,  we  often  cannot  tell  where  the  activities  of  support  firms  take  place  based  on  reports 
on  the  industry  or  on  declarations  by  the  firms  themselves.  Second,  there  is  the  question  of 
whether  the  activities  themselves  belong  inside  the  industry.  IS/IT/communications  firms  are 
particularly  troublesome  in  this  regard.  Let’s  examine,  for  example,  CACI.  It  is  a  major 
provider  of  support  services  to  the  DoD  and  to  the  intelligence  communities.  Or,  we  could 
study  Mantech.  It  builds  and  maintains  databases  that  track  potential  terrorists  and  provides 
a  range  of  other  IT-related  support  services  to  the  intelligence  communities.  If  these 
services  are  largely  performed  at  home,  should  we  define  them  as  inside  the  PM  sector? 

And  what  about  the  services  themselves — the  things  these  firms  do  certainly  appear  to  be  a 
very  different  kind  of  business  than  that  performed  by  DynCorp  and  ArmorGroup.  They  are 
involved  in  non-traditional  types  of  “warfare.”  However,  according  to  some  arguments,  if 
these  types  of  activities  reflect  the  way  conflict  is  evolving  into  the  future,  firms  like  CACI  and 
Mantech  are — arguably — critical  precursors  of  a  new  wave  of  private  defense-sector  firms. 
Should  they  be  included  in  our  sample  of  the  PM  sector,  or  left  out? 

It  is  important  to  note  that  in  reporting  on  PM-sector  demographics,  there  are 
legitimate  rationales  for  using  different  definitions  of  the  industry.  We  used  an  expansive 
definition  of  the  industry;  we  included  all  the  segments  above. 

To  conclude,  our  analysis  points  out  that  the  private  military  sector  is  by  no  means  a 
unitary  industry:  it  actually  is  an  amalgam  of  several  different  elements  that  have 
independent  drivers  and  are  developing  along  different  trajectories.  In  our  analysis,  the 
evolution  of  the  supply  side  of  the  industry  is,  therefore,  rather  complex  and  dynamic.  This 
complexity  is  partly  driven  by  a  set  of  demand  drivers  we  believe  to  be  richer  and  more 
diverse  than  is  often  acknowledged.  In  our  analysis,  the  demand  factors  driving  the  long¬ 
term  evolution  of  the  industry  involve  the  private  sector,  non-governmental  organizations 
(NGOs),  non-military  government  departments,  and  international  organizations.  Short-term 
demand  factors  are  more  military-related  and  involve  co-opting  a  sector  that  has,  in  large 
part,  traditionally  served  other  customers.  These  factors  are  dynamically  shaping  the 
evolution  of  a  heterogeneous  and  adaptive  industry  sector. 
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I  Introduction 

Services  acquisition  in  the  US  Department  of  Defense  (DoD)  has  continued  to 
increase  in  scope  and  dollars  in  the  past  decade.  In  fact,  even  considering  the  high  value  of 
weapon  systems  and  large  military  items  purchased  in  recent  years,  the  DoD  has  spent 
more  on  services  than  on  supplies,  equipment  and  goods  (Camm,  Blickstein  &  Venzor, 
2004).  The  acquired  services  presently  cover  a  very  broad  set  of  service  activities — 
including  professional,  administrative,  and  management  support;  construction,  repair,  and 
maintenance  of  facilities  and  equipment;  information  technology;  research  and  development, 
and  medical  care. 

As  the  DoD’s  services  acquisition  continues  to  increase  in  scope  and  dollars,  the 
agency  must  give  greater  attention  to  proper  acquisition  planning,  adequate  requirements 
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definition,  sufficient  price  evaluation,  and  proper  contractor  oversight  (GAO,  2002). 

Recently,  the  Director,  Defense  Procurement  and  Acquisition  Policy  (DPAP)  has  identified 
inappropriate  use  of  services  contracts  in  the  DoD  (Director,  DPAP,  2007,  March  2)  and  is 
taking  action  to  improve  contracting  for  services  throughout  the  Department  (Director, 

DPAP,  2006,  August  16).  In  many  ways,  the  issues  affecting  services  acquisition  are 
similar  to  those  affecting  the  acquisition  of  physical  supplies  and  weapon  systems. 

However,  the  unique  characteristics  of  services  and  the  increasing  importance  of  services 
acquisition  offer  a  unique  and  significant  opportunity  for  conducting  research  in  the 
management  of  the  service  supply  chain  in  the  Department  of  Defense. 

We  have  addressed  the  need  for  research  in  the  area  of  services  acquisition  by 
undertaking  a  series  of  research  projects.  Thus  far,  we  have  completed  two  research 
projects;  the  current  research  is  our  third  research  project  in  this  area. 

The  first  research  project  was  exploratory  in  nature,  wherein  we  tried  to  understand 
the  major  challenges  and  opportunities  in  the  service  supply  chain  in  the  DoD  (Apte,  Ferrer, 
Lewis,  &  Rendon,  2006).  As  a  part  of  this  research  study,  we  conducted  in-depth  case 
studies  on  acquisition  of  services  in  three  different  organizations:  Presidio  of  Monterey, 
Travis  AFB  and  the  Naval  Support  Detachment  Monterey  (NSDM).  The  major  conclusions  of 
that  research  are: 

1 .  The  Department  of  Defense’s  services  acquisition  has  continued  to  increase  in 
scope  and  dollars  in  the  past  decade.  The  GAO  found  that  since  FY  1999,  the 
DoD’s  spending  on  services  has  increased  by  66%;  indeed,  in  FY  2003,  the  DoD 
spent  over  $118  billion — or  approximately  57%  of  the  DoD’s  total  procurement 
dollars — on  services  (GAO,  2005,  March).  The  DoD  procures  a  variety  of  services, 
including  both  the  traditional  commercial  services  and  services  unique  to  defense.  In 
terms  of  amount  spent,  the  following  four  service  categories  together  represent  over 
50%  of  total  spending  on  services:  (a)  professional,  administrative,  and  management 
support  services,  (b)  construction,  repair  and  maintenance  of  structure  and  facilities, 
(c)  equipment  maintenance,  and  (d)  information  technology  services. 

2.  Presidio  of  Monterey  (POM)  has  contracted  maintenance  of  about  155  buildings  and 
structures  to  Presidio  Municipal  Services  Agency  (PMSA),  a  consortium  of  the  cities 
of  Monterey  and  Seaside.  The  PMSA  agreement  has  allowed  the  two  cities  to  apply 
their  expertise  to  routine  municipal  services  and  the  Army  to  focus  on  its  military 
mission.  Through  this  partnership  and  contract  with  PMSA,  the  POM  has  realized  a 
41%  reduction  in  expenses  when  compared  with  previous  base  operation  costs  and 
private  contracts.  We  recommend  that  the  DoD  explore  and  evaluate  the  possibility 
of  establishing  such  synergistic  contractual  relations  with  cities  adjacent  to  other 
bases  in  support  of  their  respective  operations. 

3.  Proactive  and  frequent  communications  are  essential  for  a  successful  services 
contract.  We  found  a  successful  example  of  this  at  Travis  AFB,  where  60th  CONS 
uses  Business  Requirement  Advisory  Groups  (BRAGs)  as  the  mechanism  for 
conducting  such  communications.  BRAGs  are  cross-functional  teams  made  up  of 
personnel  representing  the  functional  organizations  involved  as  customers  in  the 
services  contracts.  These  cross-functional  teams  plan  and  manage  the  service 
contracts  throughout  the  service’s  lifecycle.  As  the  DoD  increases  the  use  of 
centralized  contracting  organizations  and  regional  contracts,  the  use  of  proactive  and 
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frequent  communications  will  be  even  more  essential  for  the  successful  management 
and  performance  of  these  contracts. 

4.  Our  visits  and  interviews  at  Travis  AFB,  Presidio  of  Monterey  (POM),  Naval  Air 
Station  Whidbey  Island  (NAS  Wl),  and  the  Naval  Support  Detachment  Monterey 
(NSDM)  confirmed  the  GAO’s  finding  that:  While  the  Army’s  and  Navy’s  creation  of 
centralized  installation  management  agencies  can  potentially  create  efficiencies  and 
improve  the  management  of  the  facilities  through  streamlining  and  consolidation, 
implementation  of  these  plans  has  so  far  met  with  mixed  results  in  quality  and  level 
of  support  provided  to  activities  and  installations  (GAO,  2005,  June). 

5.  The  centralization  of  contracting  offices  and  the  use  of  regional  contracts  will  result  in 
additional  dynamics  for  the  DoD’s  acquisition  of  services.  The  Department’s  use  of 
centralized  contracting  organizations  and  regional  contracts  will  require  even  more 
proactive  and  frequent  communications  between  the  contracting  organization  and 
the  customer.  Although  it  is  still  too  early  to  assess  the  effectiveness  and  efficiency 
of  centralized  contracting  organizations  and  regional  contracts,  this  research  has 
indicated  that  centralization  and  regionalization  of  services  contracts  are  growing 
trends  in  the  DoD  and  will  significantly  change  how  services  contracts  are  managed. 

6.  Given  the  unique  characteristics  of  services  (such  as  intangibility,  co-production, 
diversity  and  complexity),  establishing  service  specifications,  and  measuring  and 
monitoring  the  quality  of  delivered  service  are  inherently  more  complex  than  with 
manufactured  goods.  Hence,  it  is  critical  to  have  onboard  a  “knowledgeable  client” 
and  the  necessary  number  of  skilled  contracting  personnel  to  define  the 
requirements  and  to  supervise  vendors  and  assure  quality  of  outsourced  services. 
The  DoD  has  been  aggressively  complying  with  OMB’s  Circular  A-76,  which  directs 
all  federal  government  agencies  “to  rely  on  the  private  sector  for  needed  commercial 
services”  (OMB,  2003).  This  has  resulted  in  dramatic  growth  in  DoD  spending  on 
services,  with  a  simultaneous  downsizing  of  the  DoD  civilian  and  military  acquisition 
workforce.  We  believe  that  the  downsizing  trend  is  not  in  sync  with  the  critical  need 
to  have  a  necessary  number  of  skilled  contracting  personnel  onboard.  This  could 
mean  that  in  the  DoD’s  outsourced  services,  either  the  needs  are  not  being  fully 
satisfied,  or  the  value  for  the  money  spent  is  not  being  realized. 

7.  As  the  DoD  acquires  more  services  than  goods,  the  acquisition  of  services  and  the 
use  of  service  contractors  are  becoming  increasingly  critical  aspects  of  the  DoD 
mission.  However,  the  management  infrastructure  for  the  acquisition  of  services  is 
less  developed  than  for  the  acquisition  of  products  and  systems.  For  example,  there 
is  a  less-formal  program-management  approach  and  lifecycle  methodology  for  the 
acquisition  of  services,  which  is  confirmed  by  the  lack  of  standardization  in  the 
business  practices  associated  with  the  services  acquisition  process.  This  results 
from  the  fact  that  the  functional  personnel  currently  managing  the  services  programs 
are  not  considered  members  of  the  DoD  acquisition  workforce  and  are  typically  not 
provided  acquisition  training  under  Defense  Acquisition  Workforce  Improvement  Act 
(, DAWIA )  requirements. 

Review  of  the  current  literature  also  shows  that  the  use  of  a  well-defined,  disciplined 
approach  and  infrastructure  for  the  management  of  projects  is  critical  for  a  project’s  success 
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in  meeting  cost,  schedule,  and  performance  objectives  (Kerzner,  2006).  In  the  absence  of  a 
well-defined  management  infrastructure,  project  teams  are  left  to  create  an  ad-hoc  approach 
to  managing  the  project.  Based  on  our  exploratory  research,  we  believe  that  this  is  the 
current  situation  in  many  DoD  services  acquisition  programs.  Both  the  lack  of  a  well-defined 
program  management  infrastructure  and  the  lack  of  a  lifecycle  approach  to  services 
acquisition  project  management  are  putting  the  success  of  these  critical  services  at  risk. 

The  risks  of  not  meeting  the  service  acquisition’s  cost,  schedule,  and  performance 
objectives  are,  consequently,  higher  in  critical  DoD  service  projects.  As  the  DoD  increases 
its  acquisition  of  services — particularly  in  light  of  anticipated  budget  cuts  and  dwindling 
resources — the  Department  must  ensure  that  its  service  acquisition  projects  are  effectively 
and  efficiently  managed. 

The  lack  of  a  developed  program  management  infrastructure  for  the  acquisition  of 
services  was  a  critical  research  finding  that  warranted  further  study.  Thus,  our  second 
research  project  was  geared  towards  studying  the  program  management  infrastructure  in 
service  supply  chain  in  the  DoD.  In  this  research,  too,  we  conducted  two  additional  in-depth 
case  studies  and  developed  a  conceptual  model  of  a  service  lifecycle  that  can  be  used  to 
analyze  and  design  the  DoD’s  services  acquisition  process.  In  our  project  report  (Apte  & 
Rendon,  2007),  we  discuss  the  program-management  approach,  identify  basic  project- 
management  concepts,  describe  how  these  concepts  are  being  used  in  the  acquisition  of 
defense  weapon  systems,  and  recommend  how  they  can  be  adapted  in  the  acquisition  of 
services  in  the  DoD. 

The  program-management  approach  essentially  consists  of  a  well-defined, 
disciplined  methodology  and  infrastructure.  The  program-management  approach  also 
includes  a  centralized,  coordinated  management  of  project  activities.  This  includes  the  use 
of  a  project  lifecycle,  integrated  processes,  designated  managers  with  project  authority, 
integrated  cross-functional  teams,  and  an  enabling  organizational  structure. 

Our  research  on  managing  the  service  supply  chain  within  the  DoD,  and  specifically 
in  the  Air  Force,  has  identified  the  following  findings: 

The  traditional  approach  to  managing  services  acquisition  does  not  include  a 
disciplined  methodology  and  infrastructure.  Nor  does  it  include  a  centralized,  coordinated 
management  of  project  activities  involving  the  use  of  the  project  lifecycle,  a  designated 
project  manager,  integrated  cross-functional  teams,  and  an  enabling  organizational 
structure. 

However,  our  research  did  identify  two  innovative  approaches  to  managing  services 
acquisition  programs.  The  Air  Education  and  Training  Command  (AETC)  approach 
incorporates  a  well-defined,  disciplined  methodology  and  infrastructure.  Through  the  use  of 
both  the  Program  Management  Flight  and  AETC  Contracting  Squadron,  the  AETC  is  able  to 
provide  centralized,  coordinated,  pre-award  management  of  services  acquisition  programs. 
And  although  in  the  post-award  management,  the  AETC  approach  does  not  maintain  an  on¬ 
site  program  manager,  it  does  maintain  an  on-site  administrative  contracting  officer.  Thus, 
regardless  of  its  success,  this  situation  has  the  potential  to  result  in  disparate  and  broken 
communications  between  all  parties  involved  in  managing  the  services  acquisition  program. 

On  the  other  hand,  the  Air  Combat  Command  (ACC)  model  for  services  acquisition 
management  using  the  Acquisition  Management  and  Integration  Center  (AMIC)  approach 
includes  a  well-defined,  disciplined  methodology  and  infrastructure,  as  well  as  a  centralized, 


ACQUISITION  RESEARCH:  CREATING  SYNERGY  FOR  INFORMED  CHANGE  -  459  - 


coordinated  program-management  approach.  The  AMIC  approach  is  unique  in  that  it 
provides  a  cradle-to-grave  acquisition  approach  to  services  acquisition  management.  This 
integrated  approach  results  in  management  efficiencies  to  include  an  effective  process 
orientation,  maximum  resource  availability  and  maximum  training  effectiveness. 

II  Current  Research  Focus 

The  objective  of  this  current  research  is  to  develop  a  more  comprehensive 
understanding  of  how  services  acquisition  is  managed  at  a  wide  range  of  military  bases 
throughout  the  Department  of  Defense.  This  current  research  is  focused  on  answering  the 
following  research  questions: 

1 .  What  types  of  services  are  typically  procured  at  military  installations,  and  what  dollar 
amount  is  annually  spent  on  these  services? 

2.  What  type  of  acquisition  strategy,  procurement  method,  and  contracts  are  used  in 
services  acquisition? 

3.  How  is  the  service  acquisition  process  managed?  What  management  concepts — 
such  as  a  lifecycle,  a  program-management  or  a  project-management  approach — 
are  used? 

4.  What  type  of  organization/management  structure  is  used  to  manage  the  services 
acquisition? 

5.  What  training  is  given  to  contract  and  project/program  management  staff? 

6.  Are  there  any  significant  differences  between  the  way  services  are  acquired  and 
managed  in  different  DoD  departments? 

Development  and  Review  of  Survey  Instrument 

The  methodology  for  this  current  research  involves  the  application  of  a  survey 
instrument  recently  developed  for  this  specific  purpose.  The  MBA  student  team  of  Compton 
and  Meinshausen,  under  the  guidance  of  Apte,  Apte,  and  Rendon,  developed  the  survey 
instrument  as  part  of  their  MBA  research  project  (Compton  &  Meinshausen,  2007).  The 
developed  survey  was  pilot  tested  for  validity  and  will  be  then  used  to  collect  additional 
empirical  data  regarding  the  current  state  of  services  acquisition  management  at  the 
installation  level  across  the  military  departments. 

The  services  acquisition  research  survey  consists  of  questions  focusing  on  specific 
demographic  data  for  each  military  department,  major  command,  region,  and  military 
installation.  The  survey  also  asks  specific  questions  related  to  the  approach,  method,  and 
procedures  used  in  the  acquisition  of  services  for  specific  categories  of  services.  The 
specific  categories  of  services  included  in  this  research  are  listed  in  Figure  1 .  These  service 
categories  are  considered  to  be  the  most  common  services  acquired  by  the  various  DoD 
departments.  The  7  service  categories  included  in  this  research  accounted  for  more  than 
$83  billion  spent  on  services  in  FY  2005  and  accounted  for  roughly  87%  of  expenditures  on 
services. 
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Service  Category 

Product/Service 
Classification  (PSC)  Code 

Professional,  administrative,  and  management  support 

R 

Maintenance  and  repair  of  equipment 

.1 

Data  processing  and  telecommunications 

D 

Medical 

Q 

Utilities  and  housekeeping 

s 

Transportation  and  travel 

V 

Maintenance  and  repair  of  real  property 

z 

Figure  1.  Service  Categories 

The  survey  instrument  includes  core  questions  related  to  the  methods  and 
procedures  used  in  the  acquisition  of  services  for  these  seven  categories  of  services. 

These  core  questions  focus  on  the  following  areas  (Compton  &  Meinshausen,  2007): 

Contract  Characteristics.  The  purpose  of  this  category  of  questions  is  to  gain  insight 
into  the  dominant  procurement  method  and  contract  type  used  in  the  acquisition  of  services 
at  the  installation.  The  characteristics  examined  in  this  section  are  degree  of  competition 
(competitively  bid  or  sole-source),  contract  type  (fixed-price  or  cost-type),  and  type  of 
contract  incentive  (incentive-fee  or  award-fee  or  award-term). 

Acquisition  Management  Methods.  The  purpose  of  this  category  of  questions  is  to 
gain  insight  into  the  types  of  management  methods  and  approaches  used  for  the  acquisition 
of  these  services  at  each  phase  of  the  contract-management  process.  For  each  of  the 
contract-management  phases,  the  survey  asks  whether  the  phase  was  conducted  at  a 
regional,  installation,  or  some  other  organizational  level.  This  core  question  category  also 
focused  on  whether  a  project-team  approach  was  typically  used  in  the  acquisition  of  the 
respective  service  category  at  the  installation  level. 

Project-team  Approach.  The  purpose  of  this  category  of  questions  is  to  explore  the 
installations  that  identified  a  project-team  approach  in  the  services  acquisition  management 
method  described  above.  The  questions  explore  the  position  of  the  services  acquisition 
project  team  leader,  such  as  a  Program/Project  Manager  or  Contracting  Officer.  This 
category  of  questions  also  explored  information  on  the  owner,  generator,  and  approving 
authority  of  the  requirement  (the  specific  service  being  acquired).  This  category  of 
questions  provides  additional  insight  into  how  a  project-management  approach  is  being 
used  in  the  acquisition  of  services. 

Service  Acquisition  Leadership.  The  purpose  of  this  category  of  questions  is  to 
explore  services  acquisitions  in  which  a  project-management  approach  was  not  dominantly 
used.  The  questions  explore  the  position  of  the  person  leading  the  services  acquisition. 

This  category  of  questions  also  explored  information  on  the  owner,  generator,  and 
approving  authority  of  the  requirement  (the  specific  service  being  acquired). 

The  last  category  of  core  questions  is  focused  on  the  use  of  a  lifecycle  approach, 
length  of  assignments  for  services  acquisition  management  personnel  staff,  use  of  market 
research  techniques,  level  of  staffing  in  services  acquisition  management,  and  level  of 


ACQUISITION  RESEARCH:  CREATING  SYNERGY  FOR  INFORMED  CHANGE  -  461  - 


training  of  services  acquisition  management  personnel.  These  questions  use  a  Likert  scale 
to  measure  the  responses. 

Finally,  the  last  category  of  survey  questions  solicits  feedback  and  any  general 
comments  regarding  the  topic  of  services  acquisition.  This  survey  instrument  will  also  allow 
the  researchers  to  collect  data  that  will  be  subsequently  analyzed  to  answer  the  research 
questions.  This  research  will  then  require  more  sophisticated  statistical  analysis — as 
discussed  in  the  next  section  of  this  paper. 

Ill  Preliminary  Hypothesis 

The  objective  of  this  study,  understanding  acquisition  of  services  at  diverse  military 
bases,  is  dependent  on  the  survey  responses.  By  designing  the  survey  in  a  specific  way,  we 
have  been  able  to  guide  the  present  and  past  direction  of  the  study  of  the  responses.  We 
analyzed  the  preliminary  results  and  recorded  the  findings.  However,  we  plan  to  further 
quantitatively  analyze  the  responses,  based  on  the  survey  currently  in  progress,  in  order  to 
provide  rigor  to  and  validation  of  our  conclusions. 

Planned  Quantitative  Techniques/Analysis 

We  plan  to  analyze  the  responses  statistically  to  find  the  proportions  of  various 
characteristics  and  management  approaches  in  principal  nodes  (depicted  in  rectangles  in 
Figure  2  and  Figure  3)  across  the  seven  categories  described  (Figure  1)  in  the  previous 
section.  In  this  analysis,  an  understanding  of  the  causes  of  predominantly  or  seldom-used 
approaches  will  lead  to  better  insight  into  the  acquisition  management  methods  of  services. 
Figure  2  and  Figure  3  describe  our  investigation  of  the  data  across  the  seven  categories. 


Figure  2.  Dominant  Procurement  Methods 
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Creation  of  an  appropriate  survey  to  guide  the  data  collection  and  answer  the 
research  questions  was  a  challenging  task.  Therefore,  the  responses  from  the  preliminary 
feedback  were  time-constrained  and,  hence,  minimal  (Compton  &  Meinshausen,  2007). 
Currently,  two  student  teams  guided  by  Apte,  Apte,  and  Rendon  (one  in  the  Air  Force  and 
the  other  in  the  Navy)  are  working  with  the  existing  survey  engine.  We  believe  these  studies 
will  result  in  sufficient  data  and  will  lead  to  substantial  statistical  analysis  offering  insight  into 
the  management  of  service  acquisition. 

The  analysis  will  explore  relations,  if  they  exist,  between  the  secondary  nodes 
(depicted  in  circles  in  Figure  2  and  Figure  3).  We  will  be  interested  in  finding  the  correlation 
between  various  independent  and  dependent  variables  that  will  represent  these  secondary 
nodes  and  other  possible  issues  discovered.  The  analysis  will  also  explore  whether  and  how 
the  dollar  amount  spent  has  any  effect  on  the  contract  characteristics  or  different 
management  approaches,  the  principal  nodes.  Based  on  the  level  of  responses  received, 
we  plan  to  simulate  the  data  if  necessary.  If  the  data  turns  out  to  be  inconclusive  in  any  of 
these  aspects,  then  that  in  itself  will  be  an  important  finding.  It  may  suggest  there  is  no 
efficient  process  in  place  for  the  acquisition  of  services — which  may,  in  turn,  lead  to  a 
recommendation  for  better  management. 
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Preliminary  Findings 

We  now  offer  some  of  our  findings  based  on  the  existing  preliminary  data.  Data 
collected  for  the  secondary  nodes  dealing  with  the  length  of  service  of  Contracting  Officer 
Representatives  and  Quality-assurance  Evaluators  shows  that  83%  of  personnel  serve  in 
their  billets  2  or  less  years.  This  is  illustrated  in  Figure  4.  We  believe  the  33%  who  serve  a 
year  or  less  imply  a  high  turnover  rate.  This  can  negatively  impact  the  quality  of  contractor 
surveillance. 


□  less  than  6  months 
H  6  to  12  months 

H  12  to  24  months 

□  24  to  36  months 
m  over  36  months 


Figure  4.  Length  of  Service 

Data  collected  for  the  level  of  staffing  and  training  (for  which  results  are  shown  in 
Figure  5)  confirm  GAO  reports  regarding  the  understaffed,  under-trained  and  under-qualified 
services  acquisition  workforce  (GAO,  2001).  These  findings  clearly  indicate  that  the 
acquisition  process  will  not  improve  until  the  situation  changes. 


5 

4 

3 

2 

1 

0 

Strongly  Disagree  Neutral  Agree  Strongly 
Disagree  Agree 


ID  There  are  an  adequate  number  of 
services  acquisition  management 
staff  positions/billets  at  this 
installation. 

B  Services  acquisition  management 
staff  positions/billets  at  this 
installation  are  adequately 
filled/manned. 

■  Services  acquisition  management 
staff  members  at  this  installation  are 
adequately  trained. 

E3  Services  acquisition  management 
staff  members  at  this  installation  are 
adequately  qualified. 
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Figure  5.  Levels  of  Staffing,  Training,  and  Qualification 

As  per  the  secondary  node  of  lifecycle  approach,  the  data  collected  shows  that  50% 
of  the  respondents  disagreed  that  a  lifecycle  approach  is  used  at  their  respective  installation 
for  both  routine  and  non-routine  services.  Based  on  this  response,  Compton  and 
Meinshausen  (2007)  reached  the  conclusion  that  “the  lack  of  a  lifecycle  approach  for  routine 
and  non-routine  services  has  the  potential  to  place  the  government  at  a  higher  level  of  risk 
due  to  improper  planning  for  the  various  phases  in  a  service’s  lifecycle”  (p.  32). 

Finally,  Figure  6  shows  that  respondents  primarily  agree  there  is  no  inconsistency 
between  requirements  identification  and  Statements  of  Work/Objectives. 

Therefore,  we  infer  that  the  cost  increase  is  not  due  to  miscommunication  of 
requirements  and  objectives.  Data  also  shows  that  respondents  agreed  that  market 
research  was  conducted  for  the  acquisition  of  services. 


Strongly  Disagree  Neutral  Agree  Strongly 
Disagree  Agree 


ID  The  entity  that  identifies  the 
requirements  in  a  service 
contract  also  writes  the 
Statement  of 
Work/Statement  of 
Objectives  (SOW/SOO)  for 

■  Market  research  is  conducted 
for  the  acquisition  of 
services. 


Figure  6.  Positive  Responses 

Thus,  based  on  the  data  collected  so  far,  our  preliminary  observations  suggest  that 
the  current  state  of  services  acquisition  management  at  the  installation  level  suffers  from 
several  deficiencies;  these  then  result  in  increasing  service  contracts.  Some  of  the  key 
aspects  are  deficit  billet  and  manning  levels  (which  are  further  aggravated  by  insufficient 
training  and  the  inexperience  of  acquisition  personnel),  and  the  lack  of  strong  project-team 
and  lifecycle  approaches.  Each  of  these  contributes  to  ineffective  and  inefficient 
management. 
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Joint  Robotics  Program 
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Introduction 

Sun  Tzu  wrote  first  about  the  importance  of  logistics  over  two  thousand  years  ago 
(Griffith,  1963,  pp.  72,  74), 67  followed  by  Von  Clauswitz  150  years  ago — who  again  echoed 
the  importance  of  logistics  to  overall  mission  success  (Greene,  1943,  pp.  136,  179)68;  now, 
logistics  is  a  Defense  Acquisition  Workforce  Improvement  Act  functional  area.  Since  Sun 
Tzu,  much  literature,  many  experiments,  lessons  learned,  and  the  DoD’s  continual 
searching  for  better  logistics  answers  have  stressed  the  continued  importance  of  getting  the 
right  things  to  the  right  place  at  the  right  time.  Much  like  human  transportation  history 
evolution — beginning  first  with  people  walking  or  running  from  point  a  to  point  b,  followed  by 
thousands  of  years  being  transported  by  real  “horse”  power,  then  automobiles,  airplanes, 
and  rockets — logistics  too  has  progressed  over  the  years:  focusing  first  on  Mass-based 
Supply,  then  Just-in-Time  Supply  Chain  Management,  and  now  on  Sense  and  Respond 
logistics. 

The  Robotic  Systems  Joint  Project  Office  (RSJPO),  an  Army-Marine  Corps  effort  that 
supplies  various  robots  to  the  AORs  of  Iraq  and  Afghanistan,  has  also  evolved  through  the 
three  logistics  methods.  During  each  approach,  many  positive  benefits  were  discovered. 
Along  with  those  benefits,  there  were  and  are  still  today  challenges  to  be  confronted  and 
overcome.  The  Robotics  Program’s  experience  and  lessons  learned  since  it  began  “real 
time”  theater  support  in  2003  can  aid  all  logistics  programs  by  exemplifying  the  better  ways 
to  provide  the  best  logistics  with  the  knowledge,  skills,  and  tools  available  today.  All 
logistics  functions,  as  shown  by  the  Robotics  Program,  can  be  provided  incredibly  fast,  quite 
inexpensively,  and  with  superior  quality  and  customer  satisfaction. 

Mass-based  Inventory 

For  many  long  years,  logistics  relied  on  provisioning  and  sparing  as  the  logical 
answer  to  supporting  any  weapons  system.  Numerous  logisticians  were  trained  in  the  art 
and  science  of  sufficient  inventory  and  spares,  which  would  keep  systems  functioning  for  the 
fielded  units  anytime  and  anywhere.  Budgets  were  predicated  against  these  projected 
numbers.  Many  logisticians  established  careers  tracking,  analyzing,  projecting,  adjusting, 
and  readjusting  Mass-based  Support  for  all  weapons  systems.  This  logistics  approach  did 
provide  weapons  system  support  that  could  function  appropriately  in  the  field  (Van  Creveld, 
1977,  pp.  206,  214).  Warfighters  (customers)  were  required  to  learn  which  spares  were 
critical  and  in  what  numbers,  while  also  trying  not  to  have  too  much  inventory  of  all  the 
wrong  things  lying  about  taking  up  needed  space  and  expending  too  much  available  budget. 


67  Sun  Tzu  gives  the  projected  costs  for  supporting  war  efforts  as  well  as  stating  logistics  for  his  time 
equated  to  60%  of  the  total  costs  incurred. 

68  Von  Clausewitz  surmises  total  war  requires  everything  relates,  including  logistics,  to  providing  the 
soldier  at  the  right  place  at  the  right  time  to  be  perfectly  effective. 
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The  robotics  program  first  began  by  utilizing  Mass  Based  logistics  to  support  the  initial 
fielding  of  162  robots.  Since  the  majority  of  suppliers  were  small  businesses  (in  DoD 
parlance,  “Mom  &  Pops”)  and  were  finding  it  difficult  to  spare  or  keep  up  with  production,  the 
Program  Manager  declared  that  a  portion  of  the  total  available  robots  would  function  as 
spares.  Central  Command  (CENTCOM)  controlled  all  robots  as  theater-provided  equipment 
(TPE),  rather  than  granting  one  of  the  Services  total  ownership.  However,  as  the  robots 
were  fielded,  Command  found  that  the  robots  worked  exceptionally  well  and  replaced 
warfighters  in  critical  danger  missions.  The  Services  would  not  release  critical  robotic 
assets  and  demanded  many  more  robotic  platforms  be  sent  into  the  field.  Very  quickly, 
CENTCOM  and  the  Project  Manager  realized  Mass  Based  Logistics  would  not  support  well 
the  customer’s  demand.  Another  logistics  approach  was  quickly  required.  What  support 
program  would  allow  the  small  businesses  to  produce,  supply,  and  keep  up  with  an  ever- 
increasing  field  demand? 

Just-in-Time 

Just-in-Time  (J-l-T)  logistics  support  promised  to  better  align  suppliers  with 
customers  in  providing  the  right  item  at  the  right  time  in  the  right  place.  J-l-T  also  promised 
to  reduce  inventories  and  spares  to  near  zero.  In  order  to  achieve  these  objectives,  quality 
would  need  to  be  more  strictly  monitored;  deliveries  would  need  to  be  timed  better;  suppliers 
would  need  real-time  communication  with  the  customers’  system  to  better  predict  when  they 
needed  to  provide  needed  items.  Production  needed  to  be  stable  so  suppliers  could  more 
easily  meet  demand  (Kotler,  1997,  pp.  214-215).  The  Robotics  program  moved  toward  J-l-T 
within  six  months  as  Massed-based  supply  could  not  keep  up.  The  Project  Manager  and 
Suppliers  gathered  data  on  which  robotic  parts  lasted  or  failed  and  how  often.  Often,  the 
same  supply  approach  (one  new  robot  for  one  damaged  or  eliminated  robot)  was  carried 
over  from  the  Mass-based  Approach.  Rebuilding  damaged  robots  grew  from  the  J-l-T 
approach.  Both  the  Project  Manager  and  the  Robotic  Suppliers  needed  faster  and  more 
accurate  information  each  day.  Tracking  robots  and  their  status  and  location  in  the  field 
became  a  pressing  point.  No  in-house  DoD  information  system  existed  to  provide  this  ever- 
increasing  communication  need.  The  Project  Manager  partnered  with  Avantix  and  T&W 
Communications  to  create  the  Catalog  Ordering  Logistics  Tracking  System  (COLTS) 
program.  The  program  utilizes  UID  formats  and  capabilities  to  provide  the  Project  Office,  as 
well  as  the  suppliers,  with  critical,  daily  information  to  meet  the  warfighters’  demands.  The 
J-l-T  approach  provided  more  accurate  robot  fielding.  Separate  warfighter  units  only 
received  robots  that  were  truly  mission  required,  rather  than  potentially  hording  robots  as 
back  ups.  The  logistic  footprint  was  reduced  as  robots  were  repaired,  rebuilt,  or  supplied  as 
needed.  The  biggest  challenge  for  the  Robotic  Project  with  J-l-T  was  caused  by 
interruptions  or  breaks  in  the  transportation  chain — disruptions  to  the  process  of  getting 
required  robots  to  their  place  of  need.  This  is  a  story  we’re  all  familiar  with  in  air  travel:  one 
weather  delay  for  the  airlines  causes  a  major  ripple  effect  to  all  airlines  and  passengers 
trying  to  get  to  the  right  place  at  the  right  time.  Once  again,  the  Robotics  Program  needed 
another  improved  logistics  solution! 

Sense  and  Respond 

This  Sense  and  Respond  section  will  flow  from  a  TAV  brief  given  at  TACOM  in 
Winter  2007  by  the  current  Robotics  Program  Manager  (Varian,  2007).  Sense  and  Respond 
logistics  arose  from  the  inability  of  J-l-T  to  completely  satisfy  the  warfighter  customer. 
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The  relationship/JPO  chart  shows  how  the  Robotics  System  Joint  Program  fits  into 
the  joint  Army/Marine  PEO.  Joint  Robot  Repair  Fielding  (JRRF)  is  just  one  of  the  areas 
necessary  for  total  Program  success. 


•  Provide  in-Theater  Support  for  Joint  Service  Theater  Provided  Equipment  (TPE)  Ground  Robots. 

•  Single  “Belly  Button”  for  OIF/OEF  Training,  Sustainment,  Assessment,  and  Accountability 

-  162  Bots  (2004)  -  1800  Bots  (2005)  -  4300  bots  (Now)  -  6000+  est’d  (2008) 

•  Pre-Deployment  Training  Sites;  JRRTs;  and  Mobile  Training  Teams  -  Joint  Reserves  (61%) 

•  4  Hour  Robot  Turn-Around  Standard  -  Leveraging  “Joint  Float  Pool  Concept” 

•  Web-Accessible  Real-time  Supply  Chain  Management  with  integrated  IUID/RFID  -  Key  step  toward 
sense  and  respond  logistics 

-  Accountability  -  Parts  Reordering  -  Reliability  Tracking  -  Trouble  Desk  Info 

•  2007  Robotic  Measures  of  effectiveness  -  Robots  save  lives 

•  25,000  (+)  IED  Missions  Conducted;  15,000  (+)  found  and  cleared  with  ground  robots;  150 
Robots  Destroyed 


r .  ,1  -i’.  '  'V  n  1 1  *■ , ,  ..v, 
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The  above  graphic  relates  the  Robotic  program  and  explains  how  the  customers’ 
needs  continue  to  rapidly  expand.  The  Program  has  instituted  no  more  than  four  hours  for 
any  robotic  replacement,  anywhere,  anytime. 


Global  Information  Grid  (GIG) 
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WEB  technology  with 
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parties  see  the  same  data 
to  manage  risk 
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to  COTS  sustainment 
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Work  Order, TM  and 
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management  results  in 
real-time  configuration 
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The  figure  above  relates  how  the  COLTS  software  program  specifically  helps 
achieve  the  overall  goals  for  Sense  and  Respond.  It  is  the  full  implementation  of  COLTS 
that  allows  IUID/RFID  to  provide  a  Total  Asset  Visibility  (TAV).  To  facilitate  Sense  and 
Respond,  TAV  and  real-time  information  flow  will  be  critical  to  sustainment  of  tomorrow.  The 
following  chart  provides  the  driving  tenets  for  all  people  involved  with  the  Sense  and 
Respond  Robotics  logistics  support. 
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1 .  Define  the  WAR-FIGHTER  :  As  the  private  or  crew  member  in  the  heat  of  it. 


2.  Understand  what  is  important  to  the  WAR-FIGHTER: 
Time 

Equipment  that  works 
Time 


3.  Define  PBL  in  a  term  the  WAR-FIGHTER  understands. 

We  structure  our  support  to  be  reactive  to  the  private  or  crew  member 
in  the  heat  of  it.  The  WAR-FIGHTER  receives  a  robot  in  4  hours  or 
less.  PERIOD 


Credibility  »  C  apability  •  Cost  * 


By  defining  processes,  the  following  charts  illustrate  how  the  Robotics  Program  office 
achieves  the  above  program  goals. 


Credibility  •  Capability  •  Cost  . 
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Pre-field ing  process 


90  days 
prior  to  field  2 


A 


SLAMIS  NSLIN  & 
MCN  are  requested 
(5  Days) 


COLTS  Catalog  built 
(1-10  days) 


SLAMIS  (PM  Push) 

Funding  Document  provided 
IDIQ  Contracts  implemented 
Sustainment  plan  provided 


System  IUID 
uploaded  to  registry 
and  marked 
1-10  days  concurrent 
With  process  A 


Supplier  &  Equipment  Data  entered  into  COLTS 
IUID  candidates  identified  (decision  chart) 
COLTS  IUID  XML  data  generated  and  uploaded 


COLTS  IUID  status  updated  and  tracked. 

Customer  (Theater)  Provided  SLAMIS  data 
Equipment  shipped  to  JRRF  for  fielding  and  TPE  update 


Equipment  fielded  \ 

to  JRRF  for  entry  concurrent  \ 
into  TPE 

Unit  draws  equipment  / 


Credibility  »  Capability  •  Cost  . 
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14 


The  Robotics  Program  has  completely  embraced  the  IUID  (Item-unique 
Identification)  method  and  is  continually  discovering  capability  benefits  from  the  warfighter 
all  the  way  to  the  supplier  and  back  again.  The  following  presents  just  some  of  these 
capabilities. 
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COLTS  (SCM)  value  to  the  RSJ 


•  Integrated  IUID  capability.  COLTS  USES  the  data  not  just  generates  the  data. 

-  Vendors  see  data  and  have  “buy  in”  with  the  IUID  process 

-  Ability  to  mark  equipment  “on  site”  virtual  IUID  NOT  REQUIRED 


•  WEB  Based  centralized  database:  There  is  no  requirement  to  “exchange” 
between  databases.  It  is  one  stop  for  common  tasks 

-  Email  notification  on  trigger  events 

-  Equipment  modification  notification  generated  and  tied  to  equipment 

-  All  stake  holders  have  access  to  the  data  and  all  see  the  same  thing 

-  Data  exportable  to  Excel™  and  data  interchange  is  possible  i.e.  DAASC,  ULLS,  etc. 

-  Reports  generation  automatically  or  data  mine  to  customize 

•  Configuration  management  up  to  15  levels 

•  Consumption  tracking: 

-  Real-time  parts  usage  and  consumption  data. 

-  Real-time  maintenance  data  (TTR,  Man-hours,  WO  processing,  etc) 

•  100%  Property  accountability: 

-  All  items  are  assets.  As  such  nothing  is  “forgotten”  items  are  always  “issued”  or 
“transferred”  but  never  forgotten 

Credibility  •  Capability  •  Cost  t 


We  could  spend  a  great  deal  of  time  relating  what  the  Robotics  program  has 
achieved  by  presenting  numerous  charts  and  graphs  of  how  improvements  have  been  made 
over  the  course  of  the  program.  But  rather  than  take  up  valuable  time  and  space,  we 
thought  just  the  bare  facts  presented  below  say  it  all... 
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Return  on  Investment 


•  A  misplaced  hyphen  cost  $280K 

•  IUID  enables  Serialized  Item  Maintenance  (SIM 
is  a  DoD  Mandate) 

-  IUID  Enables  real  time  configuration  management 

-  IUID  Saves  repair  parts  cost 

•  Aug  06-Mar  07  $29M  for  repair  parts  on  1  vendor 

•  Aug  07-Mar  08  $  5M  for  repair  parts  on  same  vendor 

•  IUID  eliminates  human  induced  error 

-  Average  human  has  a  typing  error  rate  5.47%.  For 
every  100  key  strokes  6  will  be  wrong 


Credibility  •  Capability  •  Cast  


29 


UNCLASSIFIED 


Just  the  facts  (1  Jan  07  -  1  Dec  07) 


COLTS  Supply  &  Maintenance  Data 

-  6073  Work  orders  completed 

-  26,375  maintenance  actions 

-  64,419  Inventory  events  (Parts  movement) 

-  78,467  Asset  events  (Robot  actions/movement/repair,  etc) 

-  4,816  Items  shipped 

-  64  EOD/Engineer  robots  rebuilt  from  a  destroyed  condition.  Cost  savings 
approximately  3.2  million  dollars. 


•  IUID  integration  saves  the  RSJPO  time,  money  and  ultimately  lives  on  the 
battlefield. 

-  No  more  “lost”  data  due  to  human  error 

-  Shorter  repair  cycle  time  as  a  result  of  IUID  “scan  in  &  scan  out” 

-  More  fidelity  of  data  tracked  in  COLTS  due  to  IUID  decision  process. 

-  Routine  logistics  processes  streamlined  with  IUID  and  hand  scanner. 

-  Configuration  management  integrated  with  all  SCM  actions.  SIM  is  a  reality 


•  Operational  rate  on  all  NS-E/COTS  supported  platforms  has  been  in  excess  of 
98%  since  Apr  05. 


•  In  excess  of  3307  soldiers  trained  on  robotics  operation 

Credibility  »  Capability  <  Cost  


What  more  need  we  say  about  Sense  and  Respond  logistics  and  the  benefits  it  has 
provided  to  the  warfighter? 


r .  u  '  'V  ^  1 1  l.  . ,  ..v, 
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Conclusion: 

Many  presenters  and  numerous  authors  continue  to  assert  the  rate  of  change  in  our 
era  will  continue  to  geometrically  expand.  The  Robotics  Program  since  2003  has  rapidly 
traversed  through  Mass-based,  Just-in-Time,  and  Sense  and  Respond  logistics  approaches, 
always  trying  to  improve  support  to  the  warfighter.  Each  logistics  approach  provided 
benefits  and  challenges.  Each  moved  into  another  logistics  support  scheme  built  on  the 
previous  lessons  learned  and  added  new  features — with  the  final  goal  of  reducing  the 
logistics  footprint,  expending  less  dollars,  and  providing  the  best  equipment  (which  works 
well  when  required)  to  the  right  warfighter  at  the  right  time.  The  next  chart  captures  each 
logistic  approach  and  explains  why  another  approach  was  sought. 


Approaches  to  Logistics 


Yesterday  (Mass-Based) 


Today  (Just-in-Time) 


•  More  is  better 

•  Mountains  of  stuff 
measured  in  days  of 
supply 

•  Uses  massive 
inventory  to  hedge 
against  uncertainty 
in  demand  and 
supply 

•  Mass  begets  mass 
and  slows 
everything  down 

Prime  Metric: 

Days  of  Supply 


•  On-time  is  better 

•  Inventory  is  reduced  to 
a  minimum  and  kept 
moving 

•  Uses  precise  demand 
prediction  and  static 
optimization  to  purge 
uncertainty 

•  Works  great,  except 
when  it  doesn’t 


•  Adaptive  is  better 

•  Inventory  is  dynamically 
positioned  throughout 

•  Uses  transportation 
flexibility  and  robust  IT 
to  handle  uncertainty 

•  Initial  S&R  models  look 
promising 

•  Supports  distributed, 
adaptive  ops 


Prime  Metric: 
Flow  Time 


Prime  Metric: 
Speed/Quality  of  Effects 


Credibility  *  Capability  •  Cost  - 


Even  now,  the  Robotics  Program’s  Sense  and  Respond  approach  is  not  the  final 
logistics  answer.  New  features  (active  and  passive  RFID  among  others)  are  being  tested, 
data  are  being  gathered  and  analyzed,  and  better  processes  are  being  implemented  to 
continually  improve  the  Program’s  logistics.  Other  DoD  and  industry  programs  should  take 
note  and  seek  out  people  from  this  Robotics  Program  in  order  to  discover  better  ways  to 
fully  support  the  warfighter.  The  perfect  logistics  answer  is  still  to  come. 

List  of  References 

Greene,  J.l.  (1943).  The  living  thoughts  of  Clausewitz.  New  York:  Longmans,  Green  and  Company. 

Griffith,  S.B.  (1963).  Sun  Tzu.  The  art  of  war.  London:  Oxford  University  Press. 

Kotler,  P.M.  (1997).  Marketing  management:  Analysis,  planning,  implementation,  and  control.  Upper 
Saddle  River,  NJ:  Prentice  Hall. 


ACQUISITION  RESEARCH:  CREATING  SYNERGY  FOR  INFORMED  CHANGE  -  475  - 


Van  Creveld,  M.  (1977).  Supplying  war:  Logistics  from  Wallenstein  to  Patton.  Cambridge:  Cambridge 
University  Press. 

Varian,  P.  (2007,  November  7).  Robotic  Systems  Joint  Program  Office:  Program  manager’s  brief. 
Warren,  Ml:  TACOM. 


ACQUISITION  RESEARCH:  CREATING  SYNERGY  FOR  INFORMED  CHANGE  -  476  - 


Additional  Papers: 

Logical  Decisions  for  Contracting:  Integrated  Decision 
Technology  for  Acquisition  and  Contracting 

Author:  LCDR  Roy  Garrison,  USN 
Author:  Professor  Daniel  Dolk,  NPS 
Author:  Professor  Albert  Barreto,  NPS 

Abstract 

Decision  technologies  in  the  form  of  decision-oriented  software  systems  have 
proliferated  dramatically  over  the  past  two  decades.  Most  of  these  systems  tend  to  be 
stand-alone  systems  that  are  focused  on  a  relatively  narrow  set  of  analytical  techniques  for 
solving  quite  specific  problems.  Many  applications,  however,  require  a  combination  of  these 
technologies  to  address  complex  decision-making  problems.  What  is  missing  in  the  DSS 
landscape  is  an  environment  in  which  to  create  a  DSS  Generator  that  integrates  requisite 
technologies  flexibly  and  quickly  to  construct  a  robust  application.  We  discuss  the  notion  of 
an  integrated  decision  technology  environment  (I DTE)  in  the  context  of  Federal  acquisition 
and  contracting.  Specifically,  we  show  how  the  application  of  existing  decision  support 
technologies  can  assist  Federal  Government  contracting  personnel  in  determining  which 
vendor  proposal  offers  the  best  overall  value  to  the  customer  in  competitive  solicitations. 

The  intent  is  to  establish  a  model  that,  when  implemented,  will  ensure  that  contracting 
personnel  evaluate  proposals  both  consistently  and  fairly  for  simplified  acquisition 
procedures  (SAP).  The  proposed  system,  Logical  Decisions  for  Contracting  (LDC), 
integrates  several  decision  support  technologies — including  a  weight-based  ranking  model, 
a  multi-criteria  decision  analysis  software  system,  an  expert  system,  data  mining,  and  a  data 
warehouse.  We  describe  the  data,  model,  knowledge,  and  user  interface  components  of 
LDC,  present  a  use  case,  and  show  how  virtualization  technology  can  facilitate  the 
implementation  of  this  DSS.  We  conclude  by  discussing  how  this  approach  can  be 
generalized  to  embrace  a  fuller  portfolio  of  decision  technologies  which  can,  in  turn,  address 
a  wider  array  of  more  complex  contracting  applications. 
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Abstract 

US  DoD  has  tended  to  design  Command  &  Control  (C2)  systems  without 
consideration  for  them  to  interoperate  for  synergistic  effects  since  each  is  designed  for  one 
warfighting  function.  As  systems  have  grown  biologically  into  a  System  of  Systems, 
achievement  of  mission-level  effects  has  disappointed.  Architecting  the  C2  SoS  as  a  whole 
is  improbable.  However,  capabilities-based  acquisition  requires  interoperability  certification 
based  on  delivering  a  warfighter  capability  via  SoS.  Students  at  the  Naval  Postgraduate 
School  examined  this  problem.  Their  result  is  the  Joint  Capability  Command  and  Control 
Management  (JC3M)  system.  This  paper  summarizes  their  efforts.  A  systems  engineering 
process  was  applied  to  elicit  requirements,  create  and  simulate  alternative  solutions,  and 
recommend  a  solution  with  lifecycle  cost  estimates.  The  simulation  tools  selected  to  support 
the  project  were  CORE,  to  model  function  and  data  flow;  Arena,  for  timing  and  resource 
utilization;  and  POW-ER  (Project,  Organization,  Work  for  Edge  Research),  for  organizational 
design  and  processes.  The  use  of  these  tools  to  complement  each  other  is  unique.  Results 
indicated  that  JTEM  Capability  Test  Methodology  (CTM)  was  projected  to  perform  better 
than  other  alternatives,  with  the  median  LCC.  The  final  recommendation  is  to  monitor  JTEM 
CTM  for  further  maturation  as  it  promises  improvements  in  the  utility  of  C4I  SoS  evaluations. 

Keywords:  interoperability  assessment,  modeling,  systems  engineering 

Introduction 

Across  the  US  Department  of  Defense  (DoD),  early  C4I  systems  were  designed, 
acquired,  and  fielded  independently.  Each  addressed  a  single  warfighting  function,  such  as 
logistics,  fire  support,  or  intelligence.  Over  time,  warfighting  has  grown  in  complexity, 
tempo,  and  scope.  Complex  endeavors  are  characterized  by  participants  from  not  only 
different  services  but  also  from  different  functional  areas.  They  must  respond  with  agility 
across  a  spectrum  of  action  and  across  smeared  boundaries  between  tradition  levels  of 
warfare.  The  current  scenario  requires  a  network-centric  force,  which  in  turn  requires  true 
C2  interoperability. 

Individual  C4I  systems,  most  not  designed,  acquired,  or  managed  as  a  collective 
enterprise,  are  being  integrated  as  such  and  are  forming  an  interdependent  entity — a 
System  of  Systems  (SoS) — in  which  emergent  behavior  dominates  and  capability  delivery 
cuts  across  system  boundaries.  System-level  acquisition  and  testing  only  result  in  individual 
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systems  meeting  specific  performance  requirements.  The  Joint  Interoperability  Test 
Command  (JITC)  tests  for  end-to-end  connections  “in  the  most  operationally  realistic 
environment  possible”  (rather  than  delivery  of  desired  capability)  to  assess  interoperability. 
Successful  information  exchange  results  in  “certification.”  This  is  the  baseline  system  for 
DoD  interoperability  certification.  However,  complex  interactions  of  effects  drive  changing 
configurations  of  C4I  SoS  with  no  formally  established  requirements  for  performance 
evaluation.  Capability-based  testing  of  a  SoS  is  not  well  understood.  However,  the  principle 
to  ensure  interoperability  through  testing  during  development  (National  Research  Council)  is 
still  valid. 

The  baseline  interoperability  certification  process  is  inadequate  because  it  does  not 
address  how  the  actual  SoS  supports  complex  endeavors.  Recent  revision  to  the  Joint 
Capabilities  Integration  and  Development  System  emphasizes  that  true  interoperability  is 
characterized  by  “end-to-end  operational  effectiveness  [...]  for  mission  accomplishment” 
(CJCS,  2007).  Guidance  for  writing  Capability  Development  Documents  (CDD)  requires 
Net-Ready  Key  Performance  Parameters  that  assess  “the  net-ready  attributes  required  for 
both  the  technical  exchange  of  information  and  the  end-to-end  operational  effectiveness  of 
that  exchange”  (DoD,  2004).  This  is  consistent  with  the  NATO  definition  of  interoperability 
(NATO,  2002)  and  that  proposed  by  the  Software  Engineering  Institute  (Kasunic  & 
Anderson).  Capability  Portfolio  Managers  (DEPSECDEF,  2006,  September)  and  Functional 
Capabilities  Boards  (CJCS,  2007)  play  a  role  in  capabilities-based,  cross-program 
interoperability.  Even  so,  no  system  can  assess  the  capability  of  a  SoS  requiring  integration 
of  functions  and  interfaces  across  multiple  systems.  Thus,  a  JC3M  system  is  important 
because  it  provides  a  process  for  test  planning  to  verify  true  interoperability.  It  documents 
traceability  between  capabilities  and  construction,  and  it  provides  confidence  that  the  C4I 
SoS  works. 

In  response  to  this  need,  the  Joint  Test  Evaluation  Methodology  (JTEM)  team  is 
addressing  Joint  SoS  interoperability  testing  at  the  Office  of  Secretary  of  Defense  (OSD) 
level.  Marine  Corps  Systems  Command  (MARCORSYSCOM),  the  acquisition  organization 
for  the  Marine  Corps,  is  approaching  the  issue  from  a  service  perspective. 
MARCORSYSCOM  has  tasked  the  Marine  Corps  Tactical  Systems  Support  Activity 
(MCTSSA)  to  develop  Marine  Air  Ground  Task  Force  (MAGTF)  C4I  Capability  Certification 
Testing  (MC3T),  a  methodology  for  managing  the  MAGTF  C4I  SoS  as  a  single  system. 
MC3M  will  manage  the  MAGTF  C4I  SoS  as  a  set  of  SoS-level  capabilities,  rather  than  as  a 
fixed  hardware  or  software  baseline. 

NPS  students  assigned  to  the  JC3M  project  team  adopted  a  systems  engineering 
approach  to  the  problem  of  architecting  a  C4I  SoS  assessment  system  that  will  identify 
desired  effects-based  capabilities  and  ensure  that  the  system  being  tested  meets  those 
requirements.  The  JC3M  project  sought  a  lifecycle  balanced  solution  for  existing  test 
organizations.  The  processes  can  be  utilized  by  service  and  joint  test  agencies. 

Approach  Description 

The  student  design  team  adapted  several  systems  engineering  process  models 
(Acosta  et  al,  2007)  and  tailored  them  to  this  problem.  As  illustrated  in  Figure  1,  it  begins 
with  identifying  a  customer’s  needs  and  proceeds  through  several  phases  until  a  final 
solution  is  recommended.  One  can  see  this  is  a  modification  of  INCOSE’s  SIMILAR  (state 
the  problem,  investigate  alternatives,  model  the  system,  integrate,  launch  the  system, 
assess  performance,  and  re-evaluate)  process  model  (INCOSE,  2007)  that  incorporates 
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elements  of  the  Systems  Engineering  and  Design  Process  (Paulo,  2005)  taught  at  USMA 
and  at  NPS. 


Figure  2.  A  Tailored  Systems  Engineering  Process 

During  the  problem  refinement  phase,  research  into  the  problem  space  was 
conducted,  stakeholders  were  identified  and  interviewed,  functional  decomposition  was 
started,  and  a  value  system  was  developed.  Based  on  the  preliminary  functional  analysis 
and  value  hierarchy,  several  alternatives  were  created.  Those  alternatives  were  screened, 
and  ultimately,  five  alternatives  entered  the  modeling  and  simulation  phase.  The  predicted 
performance  values  generated  by  the  models  were  used  to  objectively  analyze  those 
alternatives  by  comparing  them  to  each  other  along  with  lifecycle  cost  estimates.  The  use  of 
a  LCCE  as  part  of  the  analysis  of  alternatives  in  this  problem  domain  is  vital.  Those  testers 
and  test  planners  must  be  paid  for;  it  matters  little  if  the  final  system  provides  the  best 
solution  if  that  solution  is  unaffordable.  Finally,  a  solution  was  recommended,  along  with 
caveats.  Both  the  JTEM  project  and  MC3T  project  will  make  use  of  those 
recommendations. 

It  should  be  noted  that  this  team  did  an  excellent  job  connecting  values  identified 
early  by  stakeholders,  supported  by  a  thorough  functional  analysis.  They  integrated,  into  the 
value  hierarchy,  the  values  resulting  from  modeling  and  simulation  that  drove  the  final 
decision  process. 

Problem  Refinement  &  Functional  Analysis 

Developing  a  real  problem,  or  effective  need,  in  this  situation  proved  more 
challenging  than  anticipated.  Stating  the  central  issue  so  that  the  stakeholders  would 
receive  some  utility  from  the  final  solution  proved  slippery.  In  fact,  just  identifying  the  “right” 
stakeholders  was  a  challenge.  From  the  perspective  of  C4I  system  users,  any  process  to 
certify  a  system  is  interoperable  within  a  SoS  adds  value  when  that  certification  signifies  the 
SoS’  ability  to  support  the  complex  endeavor.  Verifying  that  it  conforms  to  technical 
standards  and  that  it  can  exchange  data  is  a  necessary,  but  not  sufficient,  prerequisite. 
Whereas,  in  the  acquisition  community,  a  program  manager  manages  resources  spent  for 
certification.  If  test  results  are  compared  to  criteria  outside  the  scope  of  his  or  her  program 
or  are  not  explicitly  stated  in  requirements  documents,  there  is  high  risk  with  little  gain.  The 
test  community,  therefore,  finds  itself  in  the  middle — being  the  honest  broker  representing 
users  while  still  adding  value  to  acquirers.  The  team  focused  on  the  test  community,  along 
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with  in-house  testers  inside  the  acquisition  community,  as  primary  stakeholders.  The  final 
list  included  the  Joint  Interoperability  Test  Command  (JITC),  Marine  Corps  Operational  Test 
and  Evaluation  Activity  (MCOTEA),  Army  Test  and  Evaluation  Command,  Navy  Operational 
Test  and  Evaluation  Force,  Air  Force  Operational  Test  and  Evaluation  Center,  MCTSSA, 
and  the  JTEM  Project  team  under  the  Director  of  Operational  Test  &  Evaluation.  As  this 
team  was  mostly  composed  of  MCTSSA  employees,  a  major  influence  was  the  new  MC3T 
project,  which  provided  an  initial  primitive  need  for  a  “system  that  defines  and  compares 
System  of  Systems  performance  measures  to  warfighter  needs  in  an  objective  and 
measurable  way”  (Finn,  2007).  They  needed  a  system  that  defined  threshold  values  for  C4I 
SoS  performance  in  operational  warfighting  terms  and  then  a  way  to  obtain  those 
performance  measures. 

The  team  examined  the  larger  context  of  the  problem  to  find  the  underlying  need. 
The  team  researched  the  most  up-to-date  interoperability  certification  and  the  latest 
direction  within  the  DoD  that  examines  realizing  desired  capabilities.  While  the  existing 
directives  and  instructions  seem  clear  in  identifying  roles  and  responsibilities  in  a  traditional 
sense,  little  light  was  shed  on  the  root  of  the  issue.  All  stakeholders  were  queried  on  how 
they  plan  a  C4I  SoS  assessment,  what  resources  they  use  to  do  so,  how  component 
systems  under  test  are  identified,  how  performance  requirements  are  codified,  how  conflicts 
are  resolved,  and  what  metrics  they  use  to  assess  their  own  performance  (Acosta  et  al., 
2007).  The  written  questions  sought  to  reveal  how  they  knew  they  succeeded  and  what 
areas  were  most  ripe  for  improvement.  The  responses  from  JTEM  and  JITC  were 
professional,  insightful  and  frank. 

A  basic  functional  hierarchy  began  to  evolve  around  the  three  major  functions: 
planning  a  C4I  SoS  evaluation,  conducting  the  evaluation,  and  reporting  results.  The 
identification  and  definition  of  performance  threshold  values  was  of  primary  concern  and  all 
stakeholders  seemed  to  be  completely  satisfied  with  their  ability  to  execute  and  report  on  an 
evaluation  event.  Therefore,  the  problem  scope  was  focused  on  the  planning  phases. 
Further  decomposition  resulted  in  a  draft  functional  model,  shown  in  Figure  2  (Acosta  et  al., 
2007). 


Figure  3.  Initial  JC3M  Functional  Decomposition 

This  project  focused  entirely  on  function  1 .0,  Plan  a  C4I  SoS  Evaluation.  Further 
functional  evaluation  identified  required  inputs  and  outputs  of  the  system,  process  activation 
and  termination,  and  evaluation  measures  for  each  of  the  lowest  level  functions. 

Eventually,  several  alternatives  were  to  be  compared  objectively.  The  basis  of  that 
comparison  was  how  well  they  achieved  the  functional  and  non-functional  requirements.  By 
combining  a  complete  functional  hierarchy  with  critical  non-functional  attributes  and 
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assigning  evaluation  measures  to  each,  a  value  system  was  created.  This  classic  systems 
engineering  paradigm  completed  the  initial  requirements  analysis  work.  A  part  of  that  value 
hierarchy — with  only  the  critical  evaluation  measures  that  were  eventually  used  in  the  final 
comparison  of  alternatives — is  in  Figure  3.  This  is  a  small  sample  of  the  information  gained 
through  the  analysis.  However,  it  is  telling  because  it  codifies  how  designers  will  know  if  we 
“got  it  right.” 


Figure  4.  Part  of  JC3M  Value  Hierarchy 

(Acosta  et  al.,  2007) 

The  lighter-colored  boxes  indicate  the  evaluation  measures  that  defined  the  needs 
for  a  set  of  modeling  tools  and  that  would  drive  the  final  analysis.  A  more  complete 
definition  for  those  elements  is  provided  in  Table  1 . 
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Table  1.  Evaluation  Measure  Details 

Design  Alternatives 

There  were  three  existing  alternatives  completed  or  in  the  final  stages  of 
development  in  response  to  the  problem  at  hand.  Additionally,  the  team  sought  to  architect 
two  additional  systems.  This  would  present  the  stakeholders  a  broad  range  of  possibilities, 
while  keeping  the  effort  required  for  modeling  and  simulation  manageable. 

The  first  of  the  known  alternatives  was  the  Federation  of  Systems  (FEDOS)  system 
used  at  MCTSSA  in  2005.  FEDOS  was  designed  to  assess  the  performance  of  C4I  systems 
when  assembled  into  the  MAGTF  C4I  SoS.  FEDOS  began  at  the  order  of  the  Deputy 
Commander  for  C4I  Integration  and  Interoperability  (C4II)  at  MARCORSYSCOM,  that 
tasked  MCTSSA  to  assess  SoS  and  systems  interoperability.  A  working  group  of 
stakeholders  in  the  system  developer  community  decided  which  systems  would  participate, 
which  requirements  were  to  be  tested,  and  the  schedule  of  events  to  include  test  planning, 
test  conduct,  and  results  reporting. 

Because  the  MAGTF  C4I  SoS  was  not  designed  in  compliance  with  an  architecture, 
there  were  no  overarching  SoS  performance  measures  or  threshold  criteria.  This  lack  of 
doctrinal  performance  criteria  meant  that  MCTSSA  test  personnel  had  to  engage  in  long, 
and  at  times,  inconclusive  negotiations  with  stakeholders  to  define  threshold  values  that 
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were  used  to  measure  performance  and  determine  if  components  passed  or  failed  the  test. 
The  MARCORSYSCOM  Product  Groups,  responsible  for  developing,  fielding,  and 
supporting  C4I  systems,  were  not  ordered  to  participate  in  FEDOS,  and  a  passing  grade 
was  not  required  for  a  milestone  decision.  It  was  perceived  as  a  no-win  situation  for  Product 
Groups:  after  a  system  had  successfully  passed  Operational  Tests  by  demonstrating 
compliance  with  system-level  performance  requirements  in  their  respective  CDD  or 
equivalent,  FEDOS  tested  component  systems  in  ways  they  had  not  been  designed  for,  but 
would  be  used  in  the  field.  The  acquisition  community’s  perception  was  that  FEDOS  was  a 
risk  with  no  off-setting  benefit.  Despite  this  shortcoming,  FEDOS  was  relatively  successful 
as  the  first  USMC  event  specifically  designed  from  the  beginning  as  a  SoS  evaluation 

Because  FEDOS  is  the  only  alternative  solution  that  has  been  used  by  a  C4I  test 
organization  for  a  true  SoS  event,  it  was  considered  the  “status  quo”  or  baseline  JC3M 
alternative  solution.  As  with  all  good  analyses  of  alternatives,  the  first  option  to  consider  is 
“do  nothing,”  or,  in  this  case,  “do  it  like  FEDOS.” 

The  second  alternative  was  MAGTF  C4I  Capability  Certification  Test  (MC3T) 
developed  at  MCTSSA  as  a  replacement  for  FEDOS.  Other  participants  in  MC3T 
development  include  the  Space  and  Naval  Warfare  Center  (SPAWAR)  Systems  Center  in 
Charleston,  S.C.,  and  the  Marine  Corps  Combat  Development  Command  (MCCDC).  More 
importantly,  representatives  of  the  MARCORSYSCOM  Product  Groups  actively  participated. 
Product  Group  representatives  defined  a  "Capabilities  Package"  complete  with  system 
requirements  and  DoD  Architecture  Framework  documents  that  depict  the  systems  under 
their  cognizance.  MCTSSA  analyzed  the  Capabilities  Package  and  produced  a 
Consolidated  Requirements  Assessment  (CRA).  The  CRA  was  an  agreement  between  the 
stakeholders  on  what  needed  to  be  tested,  the  required  resources,  and  the  Information 
Assurance  compliance  requirements.  Once  the  CRA  was  approved,  MCTSSA  produced  a 
Technical  Proposal.  The  Technical  Proposal  defined  the  technical  solution  that  the  IPT 
proposed  in  order  to  meet  the  requirements  in  the  Consolidated  Requirements  Assessment 
(CRA),  including  staffing,  C4I  systems  architecture  design,  monitoring  network  architecture 
design,  test  cases,  data  capture  and  analysis  plan,  information  assurance  plan,  and  risk 
assessment.  The  Technical  Proposal  is  confirmed,  becoming  the  Technical  Solution,  which 
makes  up  nearly  90%  of  the  Test  Plan,  includes  detailed  test  procedures  with  reference 
documentation.  The  most  promising  aspect  of  MC3T  is  that  MCCDC  and 
MARCORSYSCOM  have  developed  truly  integrated  architecture  framework  products.  The 
operational  activities  doctrinally  defined  in  the  Marine  Corps  Task  List  are  explicitly 
supported  by  specific  systems  working  together.  The  idea  that  form  should  follow  function  in 
designing  for  network-centric  effects-based  operations  is  consistent  with  the  latest  direction 
for  architectures  (DoD,  2007). 

The  third  alternative  was  JTEM’s  Capability  Test  Methodology  (CTM).  The  purpose 
of  JTEM  is  to  “develop,  test,  and  evaluate  M&P  (Methods  and  Processes)  for  defining  and 
using  a  distributed  LVC  (Live,  Virtual,  and  Constructive)  joint  test  environment  to  evaluate 
system  performance  and  joint  mission  effectiveness  [...]  focus  on  developing  and  enhancing 
M&P  for  designing  and  executing  tests  of  SoS”  (JTEM,  2007b).  Figure  4  is  an  IDEF0 
representation  of  the  CTM  process. 
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Figure  5.  JTEM  CTM  in  IDEFO 

(Acosta  et  al.,  2007) 

One  of  the  more  promising  aspects  of  JTEM’s  CTM  is  that  test  characterization 
explicitly  examines  requirements  from  families  of  CDDs  in  the  context  of  missions  based  on 
the  Universal  Joint  Task  List  (UJTL)  (CJCS,  2002)  and  Combatant  Command  standing 
operations  plans  and  orders.  More  detailed  descriptions  can  be  found  in  JTEM’s  Joint  Test 
and  Evaluation  (JT&E),  Capability  Test  Methodology  (CTM)  Method  and  Process  (M&P) 
Model  Description  (JTEM,  CTM,  M&P).  The  complexity  of  scenarios  developed  for  the  LVC 
test  environment  reflects  real-world  complex  military  action  involving  disparate  forces 
executing  closely  linked  complicated  tasks,  including  operations  other  than  war. 

Two  new  alternatives  that  offer  significant  differences  from  the  existing  systems  were 
developed.  The  classic  morphological  box  (Zwicky  process)  was  applied  and  guided  by  the 
high-level  functions  identified  earlier  and  then  used,  in  part,  to  identify  evaluation  measures. 
Nine  alternatives  were  initially  defined.  Through  several  screening  iterations  and  re- 
evaluation  against  the  root  problem,  only  two  remained:  “Systems  Capabilities  Review” 

(SCR  Alternative)  and  “Functional  Capabilities  Board”  (FCB  Alternative). 

The  Systems  Capabilities  Review  (SCR)  alternative  combines  two  of  the  original  nine 
alternatives.  It  is  composed  of  a  group  of  stakeholders:  C4I  SoS  user  representatives,  test 
agency  representatives,  system  developers  and  program  managers.  The  test  agency 
representative  chairs  the  group,  which  meets,  as  required,  to  support  a  C4I  SoS  evaluation, 
at  the  Systems  Command  level.  Inputs  to  SCR  include  source  documents  such  as 
Capabilities  Development  Documents  (CDD),  Operational  Requirements  Documents,  Test 
and  Evaluation  Master  Plans  (TEMP),  Concept  of  Operations  documents,  Joint  Integrating 
Concepts,  Joint  Operating  Concepts,  and  system  level  metrics.  First,  the  SCR  reviews  SoS 
capabilities  specifications,  examines  the  systems  engineering  artifacts  already  created  (such 
as  supporting  DoD  Architecture  Framework  documents  and  technical  performance 
measures)  and  creates  a  list  of  implied  and  stated  SoS  capabilities.  Next,  the  SCR  reviews 
system-level  documents  and  creates  a  system-level  capabilities  list.  Third,  the  SCR  maps 
system-level  capabilities  to  SoS  evaluation  measures.  The  SCR  identifies  gaps  in  the 
evaluation  measure  list  and  creates  the  balance  of  evaluation  measures  necessary  to 


ACQUISITION  RESEARCH:  CREATING  SYNERGY  FOR  INFORMED  CHANGE  -  485  - 


evaluate  the  performance  of  the  C4I  SoS.  Figure  5  illustrates  how  SCR  performs  the  JC3M 
subfunction  1.3.2  “Define  Measures.” 


SCR  Alternative 


Figure  6.  SCR  Alternative  Sub-functions 

(Acosta  et  al.,  2007) 

The  Functional  Capabilities  Board  (FCB)  alternative  relies  on  an  existing  group — the 
JCIDS  C2  Functional  Capabilities  Board — to  define  the  performance  measures  of  the  SoS. 
The  existing  role  of  FCB  is  to  perform  “organization,  analysis,  and  prioritization  of  joint 
warfighting  capabilities  within  an  assigned  functional  area”  (CJCS,  2007).  Inputs  to  the  FCB 
Alternative  include  the  UJTL  and  subsets,  Concept  of  Operations  (CONOPS) 
documentation,  acquisition  program  documentation,  and  system  trouble  reports.  The 
additional  effort  proposed  in  this  alternative  represents  an  increase  in  the  work  performed  by 
the  C2  FCB  but  is  in  the  same  functional  area  and  engages  in  the  similar  tasks.  Unlike  the 
SCR,  the  FCB  meets  on  demand,  rather  than  as  required,  to  support  SoS  evaluations.  First, 
the  FCB  will  identify  the  configuration  of  the  SoS  by  determining  the  component  systems. 
Next,  the  FCB  will  identify  the  SoS  capabilities.  SoS  CONOPS  are  reviewed  to  determine 
evaluation  measures.  Finally,  the  FCB  will  generate  the  SoS  evaluation  measure  list  for  use 
in  C4I  SoS  evaluations.  As  the  systems  under  the  cognizance  of  the  Joint  Command  & 
Control  Capability  Portfolio  Manager  are  explicitly  listed  (DEPSECDEF,  2006,  September), 
their  participation  in  this  alternative  would  be  required.  The  FCB,  under  JCIDS,  has  a  long¬ 
term  mandate,  and  provides  a  short-term  solution  to  the  lack  of  SoS  performance  measures. 
The  relationship  between  the  FCB  and  C4I  test  organizations  and  the  list  of  subtasks 
needed  to  complete  the  Define  Measures  task,  is  illustrated  in  Figure  6.  Because  the  FCB 
is  external  to  the  test  organization,  some  analysis  of  the  performance  measures  generated 
by  the  FCB  will  be  necessary.  Additionally,  it  is  understood  that  a  working  group  within  the 
FCB  would  perform  the  required  analysis. 
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Figure  7:  FCB  Alternative  Sub-functions 

(Acosta  et  al.,  2007) 

Both  of  these  new  alternatives  developed  by  the  JC3M  team  rely  on  supporting 
integrated  architectures  and  CONOPS  documentation,  in  addition  to  documentation 
normally  examined  as  part  of  C4I  interoperability  test  preparation.  The  difference  between 
these  alternatives  is  in  the  approach  taken  to  complete  process  1 .3.2  “Define  Measures”  in 
the  JC3M  Functional  Hierarchy.  The  SCR  alternative  incorporates  all  tasks  as  part  of  the 
test  planning  process.  The  FCB  Alternative  utilizes  an  external  team  that  meets  year-round 
to  provide  capability  measures  to  the  test  agency. 

Five  alternatives  had  now  been  defined  in  some  detail,  as  well  as  evaluation 
measures  to  be  used  to  compare  those  alternatives.  Only  determining  the  actual  values  or 
values  obtained  from  simulation  models  for  each  alternative  remained. 


Modeling  &  Results 

Modeling  and  simulation  were  used  extensively  in  this  project.  With  the  exception  of 
FEDOS,  no  other  alternative  under  consideration  existed.  The  only  means  to  gather 
performance  data  in  support  of  decision-making,  short  of  “building”  each  alternative,  was 
through  simulation.  It  was  the  most  cost-effective  means  to  obtain  the  required  evaluation 
measures  in  a  repeatable  and  objective  fashion.  Several  modeling  tools  were  used  to 
generate  the  necessary  data.  Figure  7  illustrates  which  tools  were  used  to  obtain  the 
evaluation  measures,  which  in  turn  supported  later  cost-benefit  analysis. 
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Figure  7:  JC3M  M&S  Supporting  Evaluation  Measures 

(Acosta  et  al.,  2007) 


Models  of  each  alternative  were  built  based  on  the  functional  architectures  already 
created.  Elements  unique  to  their  physical  instantiations  were  added.  In  other  words, 
complete  functional  models  in  IDEFO  were  created  with  Vitech’s  CORE  to  support  the 
simulation  models  built  in  Arena  and  POW-ER  (Project,  Organization,  and  Work  for  Edge 
Research).  Within  Arena  and  POW-ER,  the  attributes  that  differentiated  the  alternatives 
from  each  other —  organizational  structure,  relationships  with  external  systems,  and 
processing  of  certain  inputs — were  included.  The  IDEFO  view  of  the  systems  actually 
proved  insightful  in  terms  of  explicitly  describing  the  relationship  between  the  functions,  at  all 
levels  of  abstractions,  in  terms  of  their  inputs  and  outputs.  The  models  were  executed  by 
providing  input  to  simulate  a  system  under  test  along  with  its  supporting  information.  The 
results  of  several  iterations  with  variations  in  the  input  data  sets  were  gathered  and  used  to 
populate  the  table  of  evaluation  measures  with  raw  data.  The  “off-line  evaluation”  indicated 
the  use  of  desk-top  evaluation  by  test  and  development  community  representatives,  similar 
to  the  JTEM  Rock  Drills.  It  could  be  considered  a  kind  of  human-in-the-loop  simulation  or 
just  another  kind  of  model  or  prototype  that  has  been  used  successfully  in  this  problem 
domain  (JTEM,  2007b). 

POW-ER  is  a  project  organization  modeling  and  simulation  tool  that  integrates 
organizational  and  process  views.  POW-ER  was  developed  via  the  Virtual  Design  Team 
(VDT)  computational  modeling  research  at  Stanford  University.  POW-ER  addresses 
organizational  elements  that  impact  the  ability  to  work  effectively,  including  policies  and 
structures  (culture,  communication,  decisions,  meetings);  staffing,  hiring,  and  training  needs 
for  workforce  plans.  Using  POW-ER,  the  team  modeled  the  organizational  structure,  the 
relationship  between  individuals  within  those  organizations,  and  individual  task  allocations. 
Use  of  CORE  to  support  functional  analysis  proved  most  helpful  as  it  allowed  the  modelers 
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to  represent  the  same  functional  architecture  in  the  refined  IDEFO  models  as  a  functional 
flow  in  FFBD  format.  That  allowed  the  creation  of  PERT-like  sequencing  of  tasks  required 
when  modeling  work  processes  in  POW-ER.  POW-ER’s  ability  to  predict  and  analyze 
backlogs  proved  useful  designing  and  troubleshooting  alternative  models  because  it  allowed 
the  team  to  identify  backlogs  in  the  workflow  of  models.  The  analysis  of  backlogs  in  the 
workflow  enabled  the  team  to  identify  the  optimized  arrangement  of  tasks  and  personnel  for 
FCB  and  SCR  since  they  were  created  for  this  project.  No  such  changes  were  made  to  the 
other  alternatives.  Based  on  modeler-defined  parameters,  such  as  the  amount  of  effort 
required  for  each  task,  the  number  of  full-time  equivalents  available  with  appropriate  skills 
and  number  of  hours  in  a  work-week,  the  POW-ER  simulation  tool  can  calculate  a  project’s 
duration  based  on  simulated  duration.  Simulated  duration  factors  the  “hidden  work”  that 
traditional  Critical  Path  Method  does  not.  The  “hidden  work”  associates  an  amount  of 
rework  that  delays  into  each  task  based  upon  random  variables  described  for  each  task  by 
the  modeler.  The  simulated  duration  provided  the  number  of  days  to  plan  an  evaluation  for 
each  alternative  (Acosta  et  al.,  2007). 

Arena  is  a  commercial  tool  available  from  Rockwell  Automation.  It  provides  a 
numerical  evaluation  of  a  system  by  imitating  the  system’s  operations  or  characteristics  over 
time.  Arena  allowed  the  team  to  conduct  numerical  experiments  in  order  to  predict  the 
behavior  of  an  alternative,  given  a  set  of  conditions.  Two  evaluation  measures  required 
assessing  the  changes  in  output  as  a  function  of  the  changes  in  inputs:  Elasticity  of  Labor 
and  Elasticity  of  Duration.  Arena  allowed  the  team  to  run  simulations  on  the  alternative 
models  with  varying  sets  of  inputs.  Those  input  data  sets  represent  the  number  of  systems 
with  their  associated  documentation  that  a  SoS  test  event  would  typically  cover.  The 
baseline  data  set  was  the  group  of  systems  used  during  the  FEDOS  event.  It  included  over 
90  systems,  which  included  AFATDS,  EPLRS,  GCCS-J,  SINCGARS  and  TBMCS.  There 
were  14  SoS  capabilities  examined,  including  blue  force  common  operational  picture,  call  for 
fire,  common  logistics  and  theater  ballistic  missile  tracking.  Variation  in  the  input  data  set 
was  accomplished  by  changing  the  number  of  individual  systems,  the  number  of  old  SoS 
capabilities,  and  the  number  of  new  SoS  capabilities  under  test  for  each  data  set.  The  same 
input  data  set  was  used  for  one  run  of  each  alternative,  enabling  a  true  head-to-head 
comparison.  The  model  in  Arena  was  designed  so  that  the  subprocess  tasks  would  vary  in 
duration,  based  on  varying  the  input  systems  under  test.  Thus,  Arena  displayed  the  output 
changes  of  the  entire  alternative  process  that  corresponded  to  each  of  the  varying  inputs. 
The  output  changes  (as  a  percent  of  the  baseline),  compared  to  the  percent  change  of  the 
input  became  the  values  for  elasticity  of  duration  and  elasticity  of  labor  (Acosta  et  al.,  2007). 

The  models  were  validated  against  actual  data  from  the  FEDOS  event  of  2005. 

Since  the  original  labor  hour  timesheets  for  planning  that  event  were  available,  validating  the 
models  was  relatively  simple.  The  FEDOS  process  model  was  built  in  CORE,  which 
supported  the  more  elaborate  models  in  POW-ER  and  Arena.  Then,  the  outputs  were 
compared  to  the  appropriate  actual  data  from  FEDOS.  The  number  of  labor  hours  and 
calendar  day  predictions  from  Arena  and  POW-ER  were  within  1%  of  the  actual  values 
(Acosta  et  al.,  2007). 

Figure  8  summarizes  the  entire  simulation  process,  including  inputs  and  output 

values. 
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This  study  represents  the  first  time  these  modeling  tools  were  used  together  to 
complement  each  other.  The  simulations  predicted  key  parameters  of  each  alternative 
design.  Without  such  an  approach,  no  objective  or  repeatable  means  to  compare  the 
alternatives  against  the  requirements  in  those  areas  would  have  been  possible.  There  is  a 
high  degree  of  confidence  in  the  computer-based  measures  because  the  results  for  the 
FEDOS  models  were  validated  against  known  historical  data  and  the  other  models  used 
elements  from  the  data,  based  on  a  task  mapping  from  each  alternative  back  to  the  FEDOS 
process. 

There  were  still  two  evaluation  measures  that  could  not  be  determined  by  computer- 
based  simulation:  percent  traceable  measures  and  quality  of  planning  outputs.  The  team 
was  able  to  engage  SMEs  from  several  NAVSEA  and  NAVAIR  field  activities  to  participate 
in  assigning  a  value  for  quality  of  planning  products.  The  team  assembled  and  then 
presented  with  all  five  alternatives.  They  were  then  allowed  to  ask  questions  in  order  to 
ensure  clear  understanding  of  how  each  process  worked  along  with  built-in  limitations. 

Each  SME  responded  to  specific  questions  about  the  predicted  quality  of  planning  products 
coming  from  each  process,  with  regard  to  their  effectiveness  in  examining  interoperability 
within  a  SoS,  conformance  to  standards,  and  usability.  The  responses  were  based  on  a  4- 
point  Likert  scale  for  each  alternative.  Percent  of  traceable  measures  was  more  simple  to 
determine  once  a  key  assumption  was  accepted.  A  proxy  was  defined  as  the  number  of 
authoritative  sources  considered,  divided  by  the  total  number  of  authoritative  sources 
available.  This  assumption  is  valid  if  there  is  a  linear  relationship  (as  a  set)  between  the 
number  of  measures  created  and  the  number  of  sources  used  in  creating  those  measures. 

The  final  listing  of  the  raw  scores  is  provided  in  Table  2. 
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Percentage  of 
Traceable 
Measures 
(%> 

Days  to  Plan 
Evaluation 

(Days) 

Quality  of 
Planning 
Outputs 

(1-4  Likert 
Scale) 

Elasticity  of 
Labor 

(unit  less) 

Elasticity  of 
Duration 

(unit  less) 

Ideal  Value 

100% 

Less  is  better 

4  is  Ideal 

Less  is  better 

Less  is  better 

FEDOS 

0 

140 

3.17 

0.87 

0.87 

MC3T 

72 

121 

3.25 

0.78 

0.78 

JTEM  CTM 

92 

73 

3.42 

1.04 

0.83 

SCR 

92 

158 

3.00 

0.98 

0.98 

FCB 

88 

127 

2.75 

0.72 

0.72 

Table  2.  Raw  Evaluation  Measures 


The  extremely  short  duration  to  plan  an  event  for  the  JTEM  CTM  process  should  be 
noted.  This  is  to  be  expected  because  of  that  system’s  reliance  on  SMEs  in  so  many 
different  fields,  which  minimizes  cross-checking  with  multiple  stakeholders.  On  the  other 
hand,  the  JTEM  CTM  elasticity  of  labor  was  the  worst. 

Considering  so  many  measures,  how  could  a  single  “best”  alternative  be  found?  The 
team  chose  to  apply  classic  multi-attribute  utility  theory  (MAUT).  While  MAUT  has  its  well- 
documented  limitations,  it  presents  a  means  to  compare  the  alternatives  on  a  single 
weighted  sum  of  utilities  associated  with  each  evaluation  measure.  Raw  scores  are 
converted  to  a  value  or  utility  score;  that  value  is  then  multiplied  by  its  global  weight,  and  the 
resulting  weighted  values  are  summed  to  an  overall  value.  The  same  SMEs  who 
participated  in  the  process  to  obtain  planning,  product-quality  figures  also  participated  in  the 
process  to  determined  value  functions  and  swing  weights.  It  should  be  noted  that  this  team 
used  the  mathematically  rigorous  Wymorian  standard  scoring  functions  for  value  curves  to 
convert  raw  scores  to  utility.  Additionally,  they  were  very  precise  about  their  application  of 
swing  weights  and  rigor  of  the  analytical  hierarchy  process  to  obtain  weights  (Acosta  et  al., 
2007).  So,  the  weaknesses  inherent  in  MAUT  were  minimized  via  these  tools  and 
techniques.  The  final  total  scores  are  shown  in  Table  3. 


Percentage 
of  Traceable 
Measures 

Days  to  Plan 
Evaluation 

Quality  of 
Planning 
Outputs 

Elasticity 
of  Labor 

Elasticity  of 
Duration 

Overall 

Utility 

(0-1) 

FEDOS 

0.00 

0.04 

0.39 

0.06 

0.14 

0.63 

MC3T 

0.02 

0.05 

0.39 

0.07 

0.16 

0.71 

JTEM  CTM 

0.24 

0.06 

0.40 

0.04 

0.15 

0.89 

SCR 

0.24 

0.02 

0.37 

0.05 

0.10 

0.79 

FCB 

0.22 

0.05 

0.34 

0.08 

0.18 

0.87 

Table  3.  Overall  Utility  of  the  Alternatives 


The  last  step  in  the  process  to  consolidate  the  elements  of  the  alternatives  was  to 
create  a  lifecycle  cost  estimate  (LCCE)  for  each  alternative.  All  costs  associated  with 
development,  implementation,  operations  and  support  through  disposal  and  transition  were 
estimated.  Actual  data  from  the  FEDOS  event,  to-date  actual  costs  and  to-completion 
estimates  (directly  from  their  respective  project  managers)  for  development  of  JTEM  CTM 
and  for  development  of  MC3T  were  relatively  easy  to  capture,  once  complete  definitions  for 
those  phases  and  cost-breakdown  structures  were  developed.  Because  the  SCR  and  FCB 
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alternatives  were  similar  to  MC3T  in  scope  and  effort,  development  costs  were  based  on  the 
MC3T  numbers.  As  operations  and  support  for  such  a  system  is  dominated  by  labor  costs, 
the  annual  cost  for  each  alternative  was  based  on  applying  the  prevailing  man-hour  rates  to 
the  labor  hour  counts  from  the  POW-ER  models.  Disposal  and  transition  costs  were 
assumed  to  be  the  same  for  each  alternative  because  those  efforts  were  practically  identical 
in  terms  of  level  of  effort  and  duration.  Table  4  summarizes  the  LCCE  for  each  alternative. 


Lifecycle  Year 

Total  Cost 

Alternatives 

1 

2 

3 

4.. .9 

10 

($> 

FEDOS 

Development 

0 

0 

0 

0 

0 

0 

Implementation 

1,052,527 

0 

0 

0 

0 

1,052,527 

Operational  &  Maint. 

0 

419,497 

419,497 

419,497 

2,200 

3,908,178 

Transition  and  Disposal 

0 

0 

0 

0 

50,000 

50,000 

Total  Cost 

1,052,527 

419,497 

419,497 

419,497 

52,200 

5,010,706 

MC3T 

Development 

0 

0 

0 

0 

0 

0 

Implementation 

1,169,414 

0 

0 

0 

0 

1,169,414 

Operational  &  Maint. 

0 

525,537 

525,537 

525,537 

2,200 

4,756,500 

Transition  and  Disposal 

0 

0 

0 

0 

50,000 

50,000 

Total  Cost 

1,169,414 

525,537 

525,537 

525,537 

52,200 

5,975,913 

JTEM  CTM 

Development 

1,030,000 

2,470,000 

0 

0 

0 

3,500,000 

Implementation 

0 

0 

1,169,414 

0 

0 

1,169,414 

Operational  &  Maint. 

0 

0 

0 

558,535 

2,200 

2,253,410 

Transition  and  Disposal 

0 

0 

0 

0 

50,000 

50,000 

Total  Cost 

1,030,000 

2,470,000 

1,169,414 

558,535 

52,200 

6,972,824 

FCB 

Development 

1,021,835 

0 

0 

0 

0 

1,021,835 

Implementation 

1,301,282 

0 

0 

0 

0 

1,301,282 

Operational  &  Maint. 

0 

650,223 

650,223 

650,223 

2,200 

5,753,985 

Transition  and  Disposal 

0 

0 

0 

0 

50,000 

50,000 

Total  Cost 

2,323,117 

650,223 

650,223 

650,223 

52,200 

8,127,101 

SCR 

Development 

952,007 

0 

0 

0 

0 

952,007 

Implementation 

1,169,414 

0 

0 

0 

0 

1,169,414 

Operational  &  Maint. 

0 

624,451 

624,451 

624,451 

2,200 

5,547,811 

Transition  and  Disposal 

0 

0 

0 

0 

50,000 

50,000 

Total  Cost 

2,121,421 

624,451 

624,451 

624,451 

52,200 

7,719,232 

Table  4.  LCCE  Summary 


The  JC3M  team  determined  the  most  expensive  alternative  was  the  FCB  Alternative, 
at  a  cost  of  $8.13  million  over  the  10-year  projected  lifecycle.  The  team  calculated  the  cost 
of  FCB  as  a  cost  to  the  DoD.  While  the  senior  SMEs  who  generate  the  performance 
measures  do  not  charge  their  efforts  directly  to  a  C4I  test  organizations,  their  time  and  effort 
is  a  cost  to  the  DoD.  The  team  determined  that  MC3T  was  estimated  to  cost  approximately 
$960,000  more  than  FEDOS,  which  it  replaced.  While  this  is  nearly  a  20%  difference,  the 
increase  can  be  directly  attributed  to  the  increase  in  scope,  duration,  and  level  of  effort 
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involved  in  MC3T,  which  anecdotally  supported  the  increased  cost  of  MC3T  (Acosta  et  al., 
2007).  More  importantly,  the  development  cost  for  JTEM-CTM  is  the  largest  (its 
development  is  spread  over  several  years).  However,  the  O&S  costs  are  the  lowest.  This 
result  is  significant  because  a  test  agency  (or  test  branch  within  a  development  agency) 
deciding  between  these  options  would  incur  only  the  costs  to  implement  such  an  option  and 
then  would  reap  the  benefit  of  keeping  annual  costs  very  low. 

Recommendations 

A  complete  analysis  of  the  alternatives  based  on  the  preceding  data  was  conducted 
to  determine  the  “best”  alternative.  That  is,  which  alternative  is  projected  to  provide  the 
greatest  utility  for  the  cost?  Figure  9  summarizes  the  results.  Again,  the  utility  is  a  weighted 
sum  of  several  different  attributes,  all  tied  directly  to  the  overall  goal  of  ensuring  testing  for 
true  interoperability,  which  is  a  pre-requisite  for  any  C2  SoS  supporting  a  disparate 
networked  force. 


Life  Cycle  Cost  Estimate  ($MIL) 


Figure  9.  Utility  versus  LCC 

(Acosta  et  al.,  2007) 

The  JTEM  CTM  process  is  projected  to  perform  slightly  better  than  the  other  options 
and  maintains  a  LCCE  less  than  the  two  other  alternatives  with  the  closest  utility  scores. 
The  attributes  that  drive  this  performance  are  the  number  of  days  to  plan  an  evaluation,  the 
quality  of  planning  products  and  the  percentage  of  traceable  measures.  It  should  also  be 
noted  that  a  nearly  straight  line  could  be  drawn  between  FEDOS,  MC3T  and  FCB.  That 
leaves  the  SCR  Alternative  below  the  line  and  JTEM  CTM  above  it.  However,  the  better 
way  to  examine  this  figure  is  to  consider  an  efficient  frontier  of  utility  for  every  cost  value.  A 
linear  frontier  is  formed  by  a  line  connecting  the  points  for  FEDOS,  MC3T,  and  CTM.  Thus, 
the  FCB  and  SCR  points  are  “below”  that  line — meaning  they  are  less  efficient  and 
dominated  by  CTM. 
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It  must  be  noted  that  there  is  some  difference  in  the  confidence  we  have  in  the 
performance  measures.  Because  FEDOS  and  MC3T  were  used  in  actual  full-scale  SoS  test 
events,  their  performance  is  based  on  historical  documentation.  JTEM  CTM’s  performance 
measures  are  based  on  desk-top  simulations  called  “rock  drills,”  in  which  test  community 
personnel  exercised  certain  aspects  of  the  system  in  an  artificial  scenario.  Additionally, 
members  of  the  JTEM  team  participated  in  this  study,  which  validates  nearly  every  aspect  of 
JTEM  CTM  that  was  considered  and  confirms  the  expected  simulation  output.  The  results 
from  the  SCR  and  FCB  alternatives  were  purely  from  the  simulation.  However,  the 
simulation  was  based  on  modifying  parts  of  models  validated  through  FEDOS  data. 

With  regard  to  cost,  similar  logic  can  be  applied.  Those  numbers  from  FEDOS  and 
MC3T  are  based  on  actual  costs.  The  cost  estimates  for  the  other  alternatives,  dominated 
by  the  labor  of  annual  operations,  were  driven  by  the  simulation  output  for  number  of  labor 
hours. 

In  spite  of  the  differences  in  confidence  levels,  the  overall  results  should  be 
considered  valid.  The  JTEM  CTM  had  the  median  LCCE,  with  the  lowest  O&S  cost.  This  is 
significant  because  O&S  is  a  recurring  cost,  borne  by  every  C4I  test  organization  that 
implements  one  of  the  alternatives.  Development  costs  of  JTEM  CTM  are  the  largest 
portion  of  its  LCCE — a  nonrecurring  cost  borne  by  OSD  and  not  borne  by  any  single  C4I  test 
organization. 

Summary  &  Next  Steps 

This  team  was  the  first  to  apply  a  disciplined  systems  engineering  process  to  the 
problem  of  re-engineering  the  business  of  testing  for  C4I  interoperability  certification.  The 
JTEM  project  is  the  only  other  organization  to  examine  this  issue  from  the  perspective  of 
optimizing  a  lifecycle-balanced  solution  to  meet  explicitly  stated  and  quantifiable  needs.  No 
group  has  applied  an  integrated  set  of  computer-based  simulation  tools  to  quantitatively 
predict  the  performance  of  competing  options  and  compare  that  performance  to  lifecycle 
cost.  Knowing  that  C4I  systems  never  perform  in  a  vacuum,  but  always  interoperate  as  part 
of  a  larger  SoS,  developers  and  testers  will  benefit  from  the  results  of  this  study.  Ensuring 
interoperability  across  services  and  between  civil  authorities  and  multinational  organizations 
begins  with  an  effects-based  approach.  Only  by  testing  for  interoperability  against 
performance  measures  that  are  linked  to  desired  effects  in  the  battle-space  can  C2  SoS 
support  warfighters  engaged  in  complex  endeavors. 

Based  on  the  insights  into  the  problem  domain  and  potential  solutions,  there  are 
areas  that  need  further  study.  The  team  believed  the  C4I  acquisition  and  testing 
communities  would  benefit  from  a  dedicated  Joint  C4I  SoS  manager  to  provide  consistency 
in  an  evolving  environment.  Their  role  could  include  documenting  C4I  SoS  capabilities, 
long-range  SoS  capabilities  planning,  and  testing  requirements  management;  supporting 
developmental  and  operational  testing;  and  addressing  ad  hoc  SoS  configuration,  resulting 
from  new  threats  and  concepts  (Acosta  et  al.,  2007).  These  roles  represent  overlap 
between  the  acquisition  community  and  those  responsible  for  communicating  needed 
capabilities  to  them.  It  is  hoped  that  codifying  the  relationship  between  the  Joint  C2 
Capability  Portfolio  Manager  and  the  C2  FCB  will  be  a  move  in  this  direction. 

Next,  as  changes  to  the  SoS  configuration  are  made,  the  likelihood  of  capability 
failures  increases.  The  JC3M  team  believes  risk  management  strategies  should  be 
developed  and  applied  to  the  C4I  SoS.  The  JC3M  team’s  preliminary  list  of  risks  includes 
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the  lack  of  a  single  entity  responsible  for  SoS  performance;  the  lack  of  an  objective, 
repeatable,  and  methodical  approach  to  address  individual  system  problems  impacting  SoS 
functionality;  varied  levels  of  maturity  of  systems  within  the  C4I  SoS  architecture;  and  varied 
interfaces  between  individual  systems. 

Finally,  systems  that  are  components  of  the  C4I  SoS  have  their  capabilities  defined 
as  if  they  exist  in  a  vacuum,  and  their  impact  on  C4I  SoS  capabilities  is  generally  not 
considered.  The  DoD  C4I  SoS  acquisition  process  should  require  component  system 
sponsors  to  define  C4I  SoS  level  effects;  establish  a  funding  line  for  SoS  testing;  and 
include  SoS  effectiveness  testing  as  part  of  operational  testing  (Acosta  et  al.,  2007). 
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